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SUMMAKf 



Three sets of variables were investigated in this study: modes 
of eliciting student response to motion picture stimuli^ modes of fur- 
nishing feedback and knowledge of the correctness of the response, and 
modes of review of the presented stimuli. 



Objectives 

The purpose of the study was to compare the effectiveness of a 
sound motion picture film in sixth grade general science under the 
following conditions: 

1 . By three different methods of eliciting active learner 
response: overt constructed response, covert participation response, 
and expository no response. 

2 . By five different methods for giving the subject knowledge 
of the correctness of his response (feedback): spaced audiovisual feed- 
back with correct answer, spaced audio only feedback with correct an- 
swer, spaced audiovisual feedback without correct answer, delayed 
massed audiovisual feedback with correct answer, and no feedback. 

5. By three different methods for presenting the subject with 
a review of the content: spaced audiovisual review, spaced audio only 
review, and delayed ma,ssed audiovisual review. 

4 . For learners with different mental ability and vocabulary 
achievement characteristics. 



Procedure 

A basic instinictional sound motion picture was produced on the 
subject of "Oceanography" and modified into I5 experimental treatments 
so as to study the response/feedback variables in combination and the 
][*0Yiew variables alone. A total of h'J'J sixth grade students were ran- 
domly assigned to the I3 film treatment groups and a control group 
that did not view the film. The overt and covert response groups made 
active responses during the film showings to questions in workbooks. 
Feedback to these responses were given under the five conditions named 
above. The review groups made no responses, but viewed the feedback 
sequences. A basic film, containing no response, feedback, or isview, 
was included. Subjects were tested immediately after the presentation 
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of the films on a 67-item performance test of knowledge, comprehension 
and application. Qualitative measures of student reactions were also 
obtained. Analysis of the results was made by analysis of variance and 
covariance, factor analysis, correlation analysis, and t-test compari- 
sons. 



Results and Conclusions 

The results and conclusions are summarized below for the major 
comparisons, the learner characteristics, and the qualitative reactions. 

1 . The basic conventional film resulted in as great a learning 
gain as any of the other experimental treatments. 

2 . There was a tendency for the feedback of the knowledge of 
correctness of response to be more effective than the absence of such 
feedback. 



3. Neither overt nor covert responses were found to be superior 
to the other iinder any of the feedback conditions. 

4 . There were no significant differences in learning among any 
of the review conditions. 

5. There were no significant differences in the performance of 
subjects with different mental abilities to the different experimental 
variables . 



6. Subjects with lower vocabulary achievement interacted more 
variably with the different experimental variables than did those with 
higher vocabulary achievement. 

7. The patterns of performance on the knowledge and compre- 
hension subtests were similar to that on the total performance test, 
but the application subtest showed more variability of results. 

8. Test items previously practiced in the workbook were not 
learned appreciably better than those test items that received no 
workbook practice. 

9. All groups responded positively to the material presented, 
but they considered the films to be somewhat long and the test fairly 
easy. 
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CHAPTER I 



INTRODUCTION 



Fundamental to effective use of educational media is the design 
of the messages that pass throu^ them. Variations in message design, 
in combination with available media and known characteristics 
learners, create a complex pattern of interacting relationships that 
calls for research of conmiensurate complexity. This study was part of 
a comprehensive search for invariants in this pattern of relationships. 
Such stable factors, once identified, will become the empirical basis 
for the systematic development of principles of message desigia. 



The Problem 



In this study, controlled experimentation was used to investi- 
gate incremental presentation of a unit of instructional material on 
film. Each increment was tested immediately and the results were fed 
back to the learner, or each increment was reviewed either during or 
after the film. By thus combining several methods of presentation into 
a single experimental study, comparative assessment under controlled 
conditions became possible for selected message designs. 



Specifically, the study assessed different modes of student re- 
sponse when used with variations in feedback of knowledge of results or 
review of the material presented; compared the relative effective- 
ness of immediate feedback of knowledge of correct response and delayed 
feedback; sou^t to establish whether or not practice effect or feed- 
back following a response was operating when knowledge of correct re- 
sponse was furnished; and compared the effectiveness of spaced review 
sequences throughout the film and massed review material at the end of 
the film. Each response-feedback variable was also studied in relation 
to selected learner characteristics. 



Systematic study of specific factors in the design of messages 
for learning through audiovisual media has only recently begun. Pre- 
vious research was directed primarily to evaluative comparisons among 
the several media and techniques of their use. Now, a search for in- 
variants that contribute to the increased effectiveness of discrete 
learning units (messages) is under way. This study is part of that 

effort . 

In particular, this study has addressed itself to the measurable 
roles played in learning through media by guided response techniques 
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and the suhseq[uent feedback of knowledge of correct responses. It was 
"believed that these procedures^ until recently largely confined to 
verbal programed materials^ have demonstrable possibilities for in- 
creasing the instructional impact of projected materials. Prior re- 
search has pointed to the need for controlled manipulation of several 
related variables, alone and in combination, each step contrived to 
permit multivariate analysis of experimental and descriptive data. 

This study was designed and conducted to meet this projected need. 

The study had as its major objective the determination of the 
effects of active learner response, feedback of knowledge of the cor- 
rectness of the response; and repetitive review upon learning from a 
general science film. The specific objectives studied were: 

1, To determine the effects of mailing active overt or covert 
learner responses to workbook questions. 

2. To determine the effects of receiving feedback of knowledge 
of correct response, either immediately or when delayed, and of re- 
ceiving no feedback. 

5 . To determine the interactions of the different kinds of 
learner responses with the different kinds of feedback of knowledge of 
correct response. 

k. To compare the effects of audiovisual and. audio immediate 
review and massed delayed review of major concepts presented. 

5. To establish the learning effects of a conventional exposi- 
tory no-response film. 

6. To determine the relationships between the various stimulus 
modes and specified learner characteristics (mental ability and vocabu- 
lary achievement). 

7* determine the effects of the various stimulus modes upon 
subtest questions measuring knowledge, comprehension, or application of 
learning. 



8. To determine the effects of the various stimulus modes upon 
learning performance test questions practiced or not practiced in the 
workbook. 



9. To examine attitudinal and qualitative data that might in- 
dicate the learners ’ condition or reactions during the experiment . 

Procedurally, the problem was to produce a referent motion 
picture about the subject of oceanography to be presented in three 
learner-response modes (overt constructed response, covert participa- 
tion response, and strai^t expository with no response) combined with 
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four feedback or review modes (immediate audiovisual feedback, immedi- 
ate audio-only feedback, delayed audiovisual feedback, and no feedback 
at all). In addition to these twelve versions, a thirteenth was 
created to test the effects of presentational techniques used in the 
feedback versions. The thirteen films were shown to experimental 
groups drawn from a population of sixth-grade subjects. Effects were 
measured by tests specially developed for the experiment. A control 
group was also tested to establish prior learning of the subject matter 
used. 



Review of Related Research 



The research will be reviewed in several sections, each treat- 
ing a class of variables relatred to the problems under study: active 
learner response, feedback and knowledge of correct response, and repe- 
tition and review. 



Active Learner Response 

The eliciting of active student response in relation to in- 
structional media has been studied in a number of experiments and found 
to be effective in promoting learning (Allen, I 96 O; Cook, I 96 O; Lums- 
daine, I 965 ). Although such responses may assume different forms, this 
study directed itself to an investigation of two kinds of responses: 
overt written response and covert mental response. 



Ove rt written response . The overt foim of student response, 
either by writing answers or calling-out answers, has generally been 
favored by experimental evidence from audiovisual studies over condi- 
tions where no response was elicited. For example, Hovland, Lumsdaine 
and Sheffield (19^9), using a sound filmstrip on the teaching of the 
phonetic alphabet to Army recruits, foimd a decided superiority for 
the method in which the subjects called-out their responses to stimuli 
as contrasted with passively viewing the stimulus without overt re- 
sponse. Lumsdaine and Gladstone (195®) later substantiated these find- 
ings. Jaspen (1950), testing naval trainees in gun assembly, found 
active participation to be superior only when the film had a slow 
enou^ rate of development to permit the learners time to respond. 

Kurtz and Hovland (1955) found that elementary school children who 
verbalized at the time they observed familiar stimulus objects re- 
called more items correctly and made fewer incorrect responses than 
the non— verbalization group. Gropper and Lumsdaine ( 1961 ) compared 
the relative effectiveness of a conventional junior high school science 
television lesson and a television lesson using sequenced or programed 
material that encouraged active student response. They found that 
when the television teacher paused just prior to expected responses 
to allow students time to complete the statements begun and then "fed 
back" the correct response, learning was reliably increased. More 
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recently, Gropper (I966) compared the gains in verbal test scores by 
eighth grade students between groups that made an active response to 
programed visual science demonstrations and those that watched the 
demonstrations passively without responding. Not only did the active 
response group perform at a hi^er level immediately after the presen- 
tation, but increased its superiority significantly over the' nonactive 
response group over a four-month period and retained almost all of its 
original verbal learning. 



Covert participative response . The covert type of participation 
has been most commonly studied in connection with "question— posing 
stimuli, usually eliciting a mental response by the student. Lumsdaine, 
May and Hadsell (1958) showed that the instructional effectiveness of a 
film could be increased by splicing in "thought-provoking" questions 
designed to stimulate curiosity. Hall (1956) also reported an increase 
in the amount of learning when relevant questions were flashed on the 
screen during a silent film. However^ Lumsdaine (1958) found only a 
slight tendency for improvement through the use of rhetorical questions 
in the narration of a fifth grade science film, and Kantor (196O) fo'und 
no increase in learning when he inserted questions (both aurally and 
visually) into the content of a seventh grade science film. 



Overt versus covert response . In the several cases wher0 overt 
and covert responses to audiovisual presentations have been compared, 
the results were indecisive. Michael and Maccoby (1953) found no sig- 
nificant difference between the effects of these two kinds of responses 
when a civil defense film was stopped periodically to permit the hi^ 
school subjects to answer questions on the factual material Just 
viewed. This finding was confimed by Kanner and Sulzer (I961). Sil- 
berman (1962) reported a number of studies in programed instruction 
that showed no differences between overt and covert response modes, but 
concluded that the coveit response mode resulted in more efficient 
learning. Holland (I965) concluded, in a review of programed instruc- 
tion research, that the overt response mode co'Uld be favored over the 
covert response mode when the items were properly designed and the 
programs were long enough. However, he did note that enough doubt had 
been raised about this issue to warrant considerably more research. 



Feedback and Knowledge of Correct He suits 

Numerous studies have been reported by Allen (i960) and Lums- 
daine (1961,1963) confirming the value of "feeding-back" to tbe learner 
knowledge of results reinforcement, or confirmation of the correctness 
of his previous response to stimuli presented. No studies were found, 
however, that compared the effectiveness of different feedback chan- 
such as "audio alone" vs "audio and visual in combination. 

Also, Lumsdaine (1963) implied a need for studies that would determine 
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the functions of "immediate knowledge of correct res\ilts" with the in- 
fluence of the additional practice trial (which accompanies such feed- 
back) ruled out. 



Repetition and Review 

A number of studies have been reported by Allen (19^0) indicat- 
ing that the repetitive use of a film will result in significantly in- 
creased learning. TOien repetition and participative response were com- 
pared^ Lumsdaine^ May and Hadsell (1958) s-nd Kurtz^ Walter and Brenner 
(1950) all found that showing a film twice was about as effective as 
showing it only once with participation questions or statements in- 
serted. Althou^ research in learning showed that spaced practice is 
more effective than massed practice, Miller and Levine (1952) found 
that the massing of review material at the end of a high school science 
film was significantly more effective than spacing the review sequences 
through the film following each major topic. 
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CHAPTER II 



METHOD AND PROCEDURES 



Controlled experimentation was used to assess the combined ef- 
fects of several variables upon learning. Multiple treatments of stimu- 
lus motion picture films were presented to randomly selected groups of 
sixth grade elementary school students. Multivariate analysis was used 
to compare effects and to relate these to characteristics of the learn- 
ers. 



Experimental Design and Method 

Three sets of variables were manipulated in the study: modes of 
eliciting student response to motion picture stimuli, modes of furnish- 
ing feedback and knowledge of results of the correctness of the re- 
sponse, and modes of review of the presented stimuli. The effects of 
these variables upon the learning of the content presented in the stimu- 
lus film were compared in various combinations and are shown graphically 
in Pigure 1. 



Experimental Variables 

Learner response modes . Three different methods of eliciting 
active learner response (or no response) were designed into the experi- 
mental film treatments (shown graphically in Column 5 Pigure l): 

1. Overt constructed response ^ in -vdiich the stimulus film was 
stopped after each of eleven expository sequences, at which time the 
subjects made written responses to questions, printed in a workbook. 

These questions concerned content just viewed in the film, and the 
question was also presented on the sound track of the film. This mode 
was used in Treatment Nos. 1, 3^ 5^ 1) and 13 • 

2. Covert participation response, in which the stimulus film 
was stopped after each of the same eleven expository sequences, at 
which time the subjects were directed "to think" (rather than "to write") 
the answers to the same questions in the workbook. These instructions 
and the questions were also presented on the sound track of the film. 
This mode was used in Treatment Nos. 2, k, 6, and 8. 
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1. Treatment 
Numter 



1 



2 

5 

k 

5 

6 

7 

8 

9 

10 

IL 



12 



15 

ik 



2. Expository J. Response 4. Feedback or Review Mode 5» Expository 
Sequence Mode Sequences 



[E 



Overt 



Covert 



Audiovisual 

Feedback 



AudlO'^is, 

Feedback 



Correction 



Correction 



B...K 



B...K 






Overt 



Covert 



Audio 

Feedback 



Audio 

Feedback 



Correction 



Correction 



B...K 



B.,,K 






Overt 



Covert 



B...K 



B...K 



Overt 



B...K 



Covert 



B...K 



B...K 



Audiovisual 

Review 



B...K 






Overt 



Audio 

Review 



Audiovisual 

Feedback 



B...K 



B...K 



B...K 



Control Group (test only; no film) 

Figure 1. Graphic Description of Experimental Treatments 
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6* Massed Feedback 
or Review 



Audiovisual 

Feedback 



Audio 

Feedback 



Audiovisual 

Review 



Expository no response j in ■which, the same expository se- 
quences as in the overt and covert treatments were presented on film, 
but the film was not stopped, no questions were asked, and thus no op- 
portunity for directed response was provided. This version "was con- 
sidered to be comparable to the conventional instructional film. This 
mode was used in Treatment Nos. 10, 11, and 12. 



Feedback and knowledge of r esults modes . Five different methods 
for giving the subject knowledge about the correctness of his response 
("feedback") were also designed into the experimental film treatments 
(shown graphically in Columns 4 and 6 of Figure l) : 

1. Audiovisual feedback with correct answer, in -vdiich short 
excerpts were taken from each of the expository sequences just pre- 
sented and re shown after the student responses were made. In addition, 
the relevant page of the workbook, with the correct answers clearly in- 
dicated, was projected on the screen long enou^ for the subject to 
check and correct his response. This mode was used in Treatment Nos. 

1 and 2. 



2. Audio only feedback with correct answer , in which only the 
audio sound tracks of the same short excerpts from the expository se- 
quences just presented were played after the student responses were 
made. In this case the correct answers to the workbook questions were 
presented orally on the sound track. This mode was used in Treatment 
Nos. 3 and 4. 

3. Audiovisual feedback without correct answer j in which the 
short excerpts were taken from each of the expository sequences just 
presented and resho-wn after the student responses were made, but the 
correct answers from the workbook were not projected as was the case 
in Treatment Nos. 1 and 2. This mode was used in Treatment No. 13 • 

4. Delayed audiovisual feedback with correct answer, in which 
all of the eleven short excerpts taken from each of the expository se- 
quences were massed at the end of the film rather than being spaced 
throughout the film after each sequence . In addition, the relevant 
pages of the workbook, with the correct answers clearly indicated, were 
projected on the screen after each excerpt and long enough for the 
subject to check and correct his responses. This mode was used in 
Treatment Nos. 7 

5. No feedback, in which no feedback of any kind was made 
after the learner response, the film going immediately to the next se- 
quence. Therefore, the learner obtained no indication of the correct- 
ness of his. response. This mode -was used in Treatment Nos. 5 and 6. 



ERIC 



10 



Review. Tkiree different methods for presenting the subject 
with a review of the content of the stimulus film were designed into 
the experimental film treatments (shown graphically in Columns 4 and 6 
of Figure l)i 

1. Audiovisual review, in which the identical short excerpts 
taken from each of the expository sequences Just presented and used in 
the response/feedback modes were reshown without student response. 

Thus, without provision for student response, these excerpts served as 
a review rather than a feedback function. Ihis mode was used in Treat- 
ment No. 10. 



2, Audio only review j in which only the sound tracks of the 
same short excerpts from the expositoiy sequences Just presented were 
played immediately following the expository sequences without interven- 
ing student response. This mode was u.sed in Treatment No. 11. 

3. Massed audiovisual review, in which all the eleven short 
review excei*pts taken from each of the expository sequences were massed 
at the end of the film rather than being spaced throughout the film 
after each sequence. This mode was used in Treatment No. 12. 



Control group . The control group viewed none of the films, but 
took the performance test only. This group served as a baseline to de- 
termine whether or not there was learning from the experimental films. 



Experimental Design 

The design of the study called for the development of the I3 
experimental treatments described above, the administration of these 
treatments to experimental subjects under controlled conditions, the 
testing of performance of the subjects by means of a post-test given 
immediately following exposure to the stimulus material, and the com- 
parison of the performance data by means of appropriate statistical 
techniques. The subject matter of the stimulus film was "Oceanography," 
a subject not included in the normal sixth-grade curriculum yet compre- 
hensible to that age group. Subjects were 5-77 sixth -grade students 
from the Pasadena Unified School District (California), randomly as- 
signed to the different experimental treatments. 

Comparisons of the performance data were made for various 
combinations of the I3 treatments and a control group ("vdiich did not 
view the film, but took the performance test). These comparisons were 
made for the total groups, for hi^ vs low mental ability groups, for 
hi^ vs low vocabulary groups, for the total groups on the three sub- 
tests (knowledge, comprehension, application items), and for subtest 
items practiced in workbook vs those items not practiced. In addition, 
qualitative reactions of the subjects to the treatment modes were com- 
pared. 
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Experimental Population 

The total experimental popuiLation consisted of 477 sixth-grade 
students drawn from five elementary schools in the Pasadena Unified 
School District (California). The subject used comprised the total 
sixth-grade population of these schools. An additional group of about 
100 sixth-grade students from another Pasadena elementary school pro- 
vided subjects for the two tryouts of the performance test_, the work- 
books, and preliminary versions of the stimulus film. 

A rotational scheme was devised for assigning segments of each 
school to one of the fourteen groups. This device was necessitated by 
variations in the socioeconomic levels of the schools and the need to 
eq.ualize this factor among all the experimental groups. Using a table 
of random numbers, subjects were then assigned randomly to each seg- 
ment, comprising part of a treatment group. 



Development of the Experimental 
Stimulus Materials 

The films used in the experiment met several stringent require- 
ments: their content had to be understandable by the subjects, yet pro- 
vide sufficiently novel infom^tion to minimize effects of prior know- 
ledge and to permit development of a range of test items; the narration 
had to be meaningful without visual support in the feedback sequences 
to permit creation of the audio— only treatments; the visual elements 
had to be readily available either from existing films or throu^ use 
of art work in simple animation; and the longest edited versions could 
not run longer in time than a normal class period, including the overt 
and covert responses to workbook questions. 



Subject Matter 

Curriculum guides used in the school district were stiidied in 
search of content that would meet these criteria. The resulting list 
which included various aspects of perception, meteorology, information 
processing and transmission, navigation, and oceanography among its 28 
possibilities— was shown to several sixth-grade science teachers. Each 
independently selected oceanography, and two heads of science depart- 
ments placed particular emphasis upon physical oceanography. 

Several meetings followed with Dr. H. Bradner and his staff at 
the Scripps Institute of Oceanography in La Jolla, CalJfomia. During 
these conferences, the topics possible in physical oceanography were 
narrowed to the physical attributes of ocean ecology: in particular, 
li^t, temperature, and pressure. Later script development reduced 
these to light and temperature. 
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Scripts and Workbook 



Considerable research was undertaken to provide accurate infor- 
mational bases for the stimulus films, including extensive review of 
existing instructional films. An outline was prepared, and a first- 
draft version of film Treatment Wo. 1 (the overt response, audiovisual 
feedback treatment) was attempted. Difficulty was encountered in pro- 
viding richness of content for testing yet not exceeding the level of 
understanding of the subjects and at the same time managing to meet the 
multiple controls demanded by the many variables in the study. 

The workbook (see Appendix B) was developed in parallel with 
the script (see Appendix A) and items for the criterion performance test 
(see Appendix C). This process was primarily one of reduction from 
among a great many alternative solutions to that combination that met 
the interlocking criteria required of the study. Once the first script 
had been created, the other eleven developed rather quickly. Although 
production of the films began upon completion of the first script, 
changes were in troduced as required by the subsequent scripts and by 
the test items. 



Experimental Film Production 

Film Treatment Wo. 1 (see Figure l) contained within it aU. ele- 
ments of the other twelve films; so it was produced first. The final 
script for that film may be seen in Appendix A. A large number of 
existing instructional films and a great deal of undersea research 
footage was surveyed for materials that would fit the requirements of 
the study. Relevant materials were obtained for duplication and com- 
bined with original animated art work and original live footage photo- 
graphed especially for the experiment. 

The film was edited in several stages. A rou^ cut was made to 
organize the collected and specially prepared footage. After review on 
a picture— sound Interlocked projector, and after consequent major modi- 
fications had been mado, finer cuts were undertaken to fit a prelimin- 
ary narration. Wo optical effects, such as fades, dissolves, etc., 
were used in the film. The resulting film was then pretested with 
students in a Pasadena elementary school. Analysis of test items, data 
from interviews with students, and observations by the staff all pro- 
vided a basis for further modifications in the narration and visual 
material of the film. The revised film and test items were then tried 
out again, and again changes were made to develop or to simplify con- 
cepts found to be difficult by the subjects. For example, it was 
found that the subjects did not understand the relationship between 
the production of chemicals by bacteria at the bottom of the sea and 
the chemical feeding of plants at the surface. By animating the pat- 
tern of flow between these two levels, the concept was clarified. 



Final narration for the first film included recording of the 
brief introductory and transitional elements needed for the other treat- 
ments. Appropriate footage was then fitted to the narration^ checked 
on an interlocked projector, and then printed. In this way the requi- 
site number of treatments were created under careful control of a proj- 
ect staff producer. 



The Measuring Instruments 



Performance Test 

Description of the test. The performance test (see Appendix C) 
consisted of 67 verbal items, employing multiple- choice, true-false, 
completion fill-in, open-ended short answer, and ordering construction. 
Item types written for the performance test incorporated tasks defined 
by Bloom (1956) as Knowledge, Comprehension, and Application, '^e 
test included 32 Knowledge items, 19 Comprehension items, and 16 Appli- 
cation items. In addition, the 6? test items were categorized as 
having been practiced (19 items) or not practiced (48 items) in the 
workbook activity. 



Test development. All performance test items were developed 
throu^ a tryout-revise“-tryout cycle. Draft items designed to measure 
knowledge, comprehension, and application were prepared to sample each 
of the eleven major sequences in the films. These items were then re- 
viewed by the staff and three psychometricians.^ Devised items were 
tried out with the first draft version of Film Treatment No. 1. Scores 
were analyzed on an IBM l401 computer, using phi and biserial programs 
provided by the Testing Bureau of the University of Southern California. 
Further revisions were then made and tried out with a second sample of 
the population. Minor revisions were made to refine the performance 
test before it was finally used in the experiment. 

Reliability of the performance test, as used with the experi- 
mental group in the first tryout and as determined through use of 
Kuder-Richardson Formula #20, was .87* 1b.e reliability calculated for 
the control group was . 65 . 1 b.e mean score for the first tryout experi- 
mental group was 3 O.I 7 and for the control group (subjects who saw no 
stimulus film.) was 22.17^ giving a difference score of 8.0. Rea.iabil— 
ity of the performance test as used in the second tryout was . 9 I- with a 
mean score of 35 . 88 . The reliability estimate of the performance test 



^Dr. Paul Christensen, Human Factors Research Corporation, Santa 
Barbara, California; Dr. Philip Merrifield, Kent State University, 

Kent, Ohio; and Mr. Donald Estavan, Estavan Associates, Inglewood, 
California. 



computed from the comparable experimental version (Treatment No. l) in 
the final experiment was . 92 , and the reliability for the control group 
was - 83 . The mean score of the group that saw Film Treatment No. 1 
was '42.41, and the mean score of the control group was 25.55, with a 
difference of I 6 . 86 . 



Workbook Development 

The workbook questions (see Appendix B) were developed in 
parallel with the stimulus fiOms and the performance test measure. 
Questions were designed to promote rather than to measure learning. 
Comments from teachers during the tryout period were elicited, students 
were interviewed, answers to the questions were examined, and related 
questions on the performance test were analyzed to guide refinements 
in workbook items. 



Intelligence and Achievement Tests 

The California Test of Mental Maturity scores were used to pro- 
vide Total, Language, and Non-Language Quotient scores. The Iowa Tests 
of Basic Skills were used to provide scores for Vocabulary, Reading 
Comprehension, Language Skills, Arithmetic, and Work-Study Skills. 



F Battery 

The test comprising the F Battery were assembled to measure 
certain capabilities of students, which, when evaluated with available 
IQ and achievement scores, might improve predictions of performance with 
audiovisual materials. This test was administered to all subjects two 
weeks after their exposure to the experimental stimuli. The expecta- 
tion Was that, if the F Battery would be useful in improving predic- 
tions 'in this context, it could be utilized when forecasting perfoim- 
ance with similar stimulus materials. Explorations could then follow 
of the relationship of student abilities, not normally assessed by in- 
telligence measures, and how performance and these abilities were re- 
lated to certain combinations of instructional stimuli. 

The F Battery was constructed by selecting a number of tests 
identified as being factorially unique by the research staff of the 
Psychological Laboratory of the University of Southern California 
(Guilford & Hoepfner, 1963 ) . All the tests selected were based upon 
the "Structure of Intellect" model (Guilford, 1955)* This model pro- 
vided for a logical and systematic placement of every "known" intel- 
lectual ability. The tests chosen had been identified as being re- 
lated to one of some 55 different intellectual abilities ascertained 
through the use of multiple-factor analysis by the Aptitudes Project 
of the Psychological Laboratory. The test battery itself had been 
assembled for use in an earlier experiment (Allen, Filep and Cooney, 

1967). 
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In addition to factorial uniqueness, the tests selected were 
based upon: (l) prior use and reliability with a preadolescent/ 

adolescent population, (2) dissimilarity to subtests of the IQ and 
achievement measures, and ( 3 ) relationships to tasks required in re- 
sponding to the stimulus materials provided in the programed sequences. 

Tests were selected that measured the student’s ability to iden- 
tify, compare and contrast, and construct graphic figures. The spe- 
cific tests that were selected measured the student’s ability: to 
judge quickly and accurately units of figural information as being 
identical or dissimilar, to produce many simple figures that conformed 
to given specifications, to produce changes in figures that altered 
the meaning significance or use of elements, to classify the same 
items of figural information in different ways, or to place events in 
correct sequence. 



Semantic Differential 



The Semantic Differential instrument consisted of eleven 7- 
point scales defined by adjective pairs representing three factors: 
Evaluative, Potency, and Action (see Appendix D). The rationale, con- 
struction, and analysis of this instrument has been described in Os- 
good, Suci, and Tannenbaum (1957) • 



Evaluative 

good-bad 

work-play 

cruel-kind 

plea sant-unpleasant 



Potency 

strong-weak 

heavy-light 

feminine-masculine 



Activity 

fast- slow 

repet it ious-var ie d 

loud- soft 

active-passive 



The concept, "THE STQEY YOU SAW ABOUT THE EGO-SYSTEM " was to 
be rated on each of the eleven scales. The scales were printed on mask 
sense forms so that scoring machines could be used to transfer the data 
to punched cards. 



Student Reaction Questions 



A student reaction scale, consisting of five 5-point scales, 
and three open-ended questions were constructed to elicit ratings and 
reactions relative to the students’ perceptions of the length of the 
film, liking for the film, learning from the film, test difficulty, 
helpfulness of the workbook in learning the material, and trouble 
understanding the film, (see Appendix D). 
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Conduct of the Experiment 



Schedules and procedures were worked out in detail with the 
administrative staff of each school and then were reviewed with the 
teachers of the classes involved in the experiment. At the times of 
the experimental runs, the subjects met in their home rooms and were 
then taken to the appropriate experimental rooms as detemnined by their 
random assignment. The experimental rooms were regular classrooms set 
up for film projection. Students not scheduled for the first presenta- 
tion were supervised on the playground or in a study hall, whichever 
contingency disturbed their normal schedules the least. All subjects 
in the experimental rooms were presented with the instructions, the 
stimulus films and workbook exercises, and then tested immediately with 
the performance test. Following the administration of the experiment, 
these subjects were then moved to the playground or study hall, and the 
second experimental group was brou^t to the experimental rooms. Con- 
tact was avoided between these two experimental sections by keeping 
them separated during the passing periods. 

Two weeks later the F Battery, semantic differential measure 
and the student reaction questions were administered to all the sub- 
jects, either in their own classrooms as intact class groups or in 
combined groups in cafetoriums, whichever was most convenient at the 

school. 



Preparation of Data and 
Statistical Analysis 



Recording the Data 

All quantitative data were to be analyzed by computer; so ap- 
propriate formats were set up that would simplify detection and correc- 
tion of errors in scoring of tests and in transfer of descriptive data 
from school records to the nuhxoered cards used in the analysis. 



Du^-ing the experiment, subjects marked their answers directly 
on the test booklets. ^ These booklets were then grouped by treatment 
and scored twice independently on sensescore sheets. Guide keys (see 
Appendix C) were provided to each of two scorers to indicate acceptable 
and unacceptable answers. The two sensescore sheets for each test 
booklet were then compared, and differences were resolved for all tests 
by a third individual. Percentages of differences ranged among experi- 



^It was found during the tryouts that subjects had serious dif- 
ficulty rel*=>bing numbered items accurately to the sensescore sheets; 
so subjects marked answers directly in the test booklets. 
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mental groups from a low of 1.5^^ the group that saw film number 9 
to a high of 3*2^ for the group that saw film number 2. The mean per- 
centage of difference was 2.3^ for all groups, the percentage of all 
scored items that required interpretive resolution of differences be- 
tween scorers. 

Estimates of abilities and general achievement were provided 
from school records ( Iowa Tests of Basic Skills and the California Test 
of Mental Maturity ). Numbers assigned to students for purposes of 
randomization were used throu^out the analysis to preserve the anonymi- 
ty of the subjects and the integrity of their sdiool records. IBM cards 
wei’e punched manually from the lists of student data. The multi-part 
P Battery, used to measure selected aptitudes, was also scored once by 
each of two different scorers. Initial differences were resolved in 
conference with a third party to establish reasonable correct criteria. 
This approach eliminated most of the conflicting assessments. The re- 
mainder were brought to the third party for decision. Scores were 
transferred to sensescore sheets for mechanical transfer to IBM cards. 

All sensescore sheets, including those for the semantic differ- 
ential, were processed by the University Testing jjureau. Resulting 
cards were then sorted and analyzed by machine for descriptive charac- 
teristics (listing, ranges, means, frequency, counts, etc.). These re- 
sults provided a basis for ''cleaning" the cards of errors. This cycle 
was repeated until all observable errors were removed. 



Statistical Techniques 
and Computer Program 

A series of descriptive analyses were undertaken to establish 
the quality of the data, to identify characteristics of the population 
and interrelationships among selected variables, and to provide a basis 
for selecting covariates to be used in the comparative analyses. All 
these analyses were undertaken on the Honeywell 800 computer, operated 
by the staff of the Computer Sciences Laboratory, University of Southern 
California. All computer programs used were created or adapted for use 
on this machine by senior staff members of the Laboratory. In particu- 
lar, factor analytic and covariance programs used were adapted versions 
of the BMD03M and BMD05V programs, respectively, described in Dixon 

(1965). 



The principal components factor analysis used provided means 
and standard deviations, a correlation matrix, and a factor matrix use- 
ful in the selection of covariates for later analyses. An orthogonal 
rotation to the varimax criterion was performed on six factors. 

Analyses of variance and covariance were undertaken to establish 
the statistical differences among the experimental variables and to ad- 
just the test scores for differences among the groups in vocabulary 
achievement and non-language I.Q. 
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CHAPTER III 



RESULTS 



Test results were analyzed for total post-test performance, for 
post-test performance ty high vs low mental ability groups, for post- 
test performance of hi^ vs low vocabulary ability groups, for the total 
groups on the three subtests (imowledge, comprehension, application 
items), and for total groups on the subtest items practiced in the work- 
book vs those items not practiced. In addition, qualitative responses 
on the semantic differential measure and reactions to the presentation 
were compared for all groups. 



Descriptive Analy ses 



Homogeneity of Variance 

Frequency distributions, associated polygons, and plots were 
computed to provide an estimate of homogeneity of variance among groups. 
Kurtosis of the distribution "of raw scores was found by calculation to 
be significant at the .05 level. 3 However, after evaluation of this 
estimate, in view of the graphic descriptions mentioned above, and in 
view of current estimates of its effects in the analyses to be under- 
taken, sufficient confidence was held in the data to proceed (Kerlinger, 

1964, pp. 258-259). 



Selection of Covariates 

Covariates for the analysis of covariance were selected by per- 
forming correlation and factor analyses of the test scores. Three cri- 
teria were used in the selection of the covariates. The first required 
that the covariates selected be meaningful in the context of the study 
to members of the educational community who mi^t be using the reported 
results. Second, since only two covariates were to be used, it was 



^Calculated on the Honeywell 8OO computer with a program pre- 
pared by Richard Wolf and Leopold Klopfer, entitled "Test Scores and 
Statistical Analysis 2," of the Graduate School of Education, University 
of Chicago. 
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important that one covariate have a large loading on one factor and the 
other covariate have a large loading on another factor. Thirds these 
two covariates also had to have a low correlation with each other and 
a significant correlation with the total performance post-test. 

The covariates selected were Non-Language IQ and Vocabulary^ 
achievement. Both of the covariates selected were measurable by exist- 
ing tests, thus meeting the first criterion. Examination of Table 1 
shows that Non-Language IQ had a hi^ loading (.476) on Factor 5 and 
that Vocabulary achievement had a high loading (.866) on Factor 1. 

Table 21 in Appendix E shows these two covariates to have a .4l corre- 
lation with each other, which was moderate within the range of possible 
covariates to be considered. Non-Language IQ correlated .45 with the 
total performance post-test, and Vocabulary achievement correlated .j 6 
with that test. 



TABLE 1 

rotated factor loadings used as a basis for selecting covariates 

APPLIED IN TEE ANALYSIS OF COVARIANCE (N = J> 3 l) 



Variable 






Factors 






1 


2 


5 


4 


5 


6 


vocabulary 


.866 


.161 


.198 


.028 


.166 


-.157 


Reading 


.855 


.115 


.528 


-.115 


.182 


-.105 


Language Skills 


.660 


.256 


.496 


.595 


.113 


.065 


Work Study 


.569 


.598 


.545 


-.020 


.265 


-.027 


Arithmetic Skills 


.559 


.507 


.580 


-.097 


.067 


.091 


Language IQ 


.647 


.175 


.446 


.100 


.241 


-.087 


Non-Language IQ 


.277 


o207 


.476 


.065 


.596 


.065 



Correlation of Variables 

Correlations listed in Appendix E showed the performance post- 
test as being highly correlated with the achievement test scores, with 
IQ sccres, and with the total F Battery score. The parb scores (know- 
ledge, comprehension, application) of the performance post-test also 
exhibited a hi^ positive relationship to the intelligence, achieve- 
ment, and total F Battery scores. On the other hand, sex had a -.11 
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correlation with the post-test score. The correlations "between achieve- 
ment scores and IQ and F Battery scores tended to be fairly high, and 
the correlations between the IQ scores and the total F Battery score 
was in the me dim range. 



Analysis of Total Performance 

Mean scores and standard deviations for the total performance 
post-test and for the Knowledge, Comprehension, and Application sub- 
tests are presented in Table 2. Examination of the total imcorrected 
performance post-test scores for the thirteen treatment groups shows a 
range from 35.80 to 46.86, the Overt Response/Audiovisual Feedback With- 
out Correction being the most effective and the Delayed Massed Review 
the least effective. The subtests followed the same pattern of effec- 
tiveness. 

Comparative analyses among different combinations of the treat- 
ment groups were conducted in order to meet the specified purposes of 
the study. Means, uncorrected and corrected, will be presented with 
each analysis to show the differential effects of the covariates. 



Response and Feedback Modes 

Six separate analyses were made of the effects of the different 
learner response and feedback variables upon post-test performance. 

These analyses combined the response and feedback variables in different 
groupings in order to determine their individual and Interacting effects. 

All ei^t treatments in which overt or covert responses were 
made to the experimental stimuli were compared under the four different 
feedback conditions as represented in the following design matrix (with 
the filn treatments indicated by their nmbers): 



FEEDBACK 


RESPONSE 


Overt 


Covert 


AV Immediate 


1 


2 


Audio Immediate 


5 


4 


No Feedback 


5 


6 


AV Delayed 


7 


8 



A two-way analysis of covariance was performed, on these data 
with Non-Language IQ and Yocahiilary used as covariate s. Table 5 pre- 
sents the unadjusted and adjusted performance post-test means, the 
means for the two covariates, and the results of the analysis of co- 

variance . 



TABLE 5 

RESPONSE VS FEEDBACK (ANALYSIS OF COVARIANCE) 
( Covariate SI Non— Language IQ and Vocabulary) 





N 


Criterion 


Covariate X 


Adjusted 

Criterion 

X 


X 


a 


Non-Lang. 

IQ 


1 

Vo cab . 


OVEPiT RESPONSE 














1 . AV Immediate 


54 


42o91 


12.90 


105.05 


68.55 


45.06 


5. Aud,io Immediate 


55 


45.65 


9.01 


106.05 


72.60 


41.47 


5. No Feedback 


51 


4 l. 6 l 


10.59 


112.26 


69.32 


4o.6o 


7. AV Delayed 


29 


44.14 


9.95 


117.79 


72.69 


40.88 


COVERT RESPONSE 














2 « AV Immediate 


54 


42.27 


8.59 


106.55 


67.65 


42.66 


4 . Audio Immediate 


56 


44.75 


10.08 


102.64 


71.78 


45.52 


6 . No Feedback 


29 


58.04 


12.85 


110.17 


66.90 


58.49 


8 . AV Delayed 


55 


40.76 


10.80 


105.67 


67.79 


4 l.l 4 



Feedback 

Response 

Interaction 



Within 



df 

5 

1 

5 



251 



ss 

410.89 

.67 

151.96. 



11605.48 



m 


F 


Pr ob . 


156.96 


2.96 


< 0.05 


.67 


0.01 


— 


45.99 


0.95 


— 


46.24 
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Uie analysis of covariance F-test ^owed a .05 level of signifi- 
cance among the feedback variables, but no differences were found be- 
tween the response modes or the feedback-re^onse Interactions. The 
two active response modes (overt and covert) with either 
audio feedback of knowledge of correct response were the most effective 
modes of presentation, and the mode furnishing no feedback was the 
least effective. 

The results were separately compared in one-way analyses 
variance for each of the active response conditions (overt and covert) 
with the Inclusion of the Expository No Response group (Film Treatment 
9). The results of these two analyses are shown in Tables l- and iMO 
simificant differences among the five treatment groups were f o\md f or 
the subjects maJting "overt" responses to the workbook exercises. How- 
ever, a difference significant at the .05 level was found for ‘t^se 
subjects making "covert" responses to the workbook exercises. The 
treatment giving no feedback to the covert response was the least ef- 
fective of the five treatments. Despite the fact that the Exposl o^ 

No Response treatment contained no provisions for either active student 
response or feedback, it appeared to be as effective as any of the 
other experimental treatments. 



TABLE k 



OVERT RESPONSE AND NO RESPONSE (ANALYSIS OF COVARIANCE) 
(Covarlates: Non— Language IQ and Vocabulary) 









Criterion 


Covariate X 


Adjusted 

Criterion 

X 






N 


X 


O’ 


Non-Lang. 

IQ 


Vocab . 


1. 


AV Immediate 


54 


42 o 91 


12.90 


105.05 


68.55 


44.71 


5. 


Audio Immediate 


55 


45.65 


9.01 


106.05 


72.60 


43.03 


5. 


No Feedback 


51 


41.61 


10.59 


112 o26 


69.52 


42.18 


7. 


AV Delayed 


29 


44.14 


9.95 


117.79 


72.69 


42.34 


9. 


Expository 
No Response 


50 


45.57 


11.17 


104.45 


72.13 


43.57 





df 


SS 


MS 


F 


Prob. 


Overt X No Response 


4 


128.54 


52.14 


.72 


— 


Within 


152 


6786.68 


44.65 







TABLE 5 



COVERT RESPONSE AND NO RESPONSE (ANALYSIS OF COVARIANCE) 
(Covariates: Non-Language IQ and Vocabulary) 







Criterion 


Covariate X 


Adjusted 

Criterion 

X 




N 


X 


cr 


Non-Lang. 

IQ 


Vo cab. 


2 . AV Immediate 


54 


42.27 


8.59 


106.55 


67.65 


42.87 


4 . Audio Immediate 


56 


44.75 


10.08 


102.64 


71.78 


45.88 


6 . No Feedback 


29 


58.04 


12.85 


110.17 


66.90 


58.68 


8 . AV Delayed 


55 


40.76 


10.80 


105.67 


67-79 


41.55 


9. Expository 

No Response 


50 


45.57 


11.17 


104.45 


72.13 


42.29 





df 


SS 


MS 


F 


Prob. 


Covert X No Response 


4 


471.54 


117.89 


2.62 


< .05 


Within 


155 


6987.90 


45.08 







A tvo-way analysis of covariance was also performed^^to compare 
the perfoimance of the four groups that received "immediate" audio- 
visual or audio feedback to either overt or covert responses, as repre- 
sented in the following design matrix: 





BESPOKSE 


FEEDBACK 


Overt 


Covert 


AV Immediate 


1 


2 


Audio Immediate 


5 


4 



The results of the analysis are shown in Table 6 . No significant dif- 
ferences were found between the two active response modes, between the 
two feedback modes, or for the interactions among these variables under 
conditions of "immediate" feedback of knowledge of correct response 
with the correct answers shown. 
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Another tvo-vay analysis of covariance was perfc vined to compare 
the performance of those groups receiving "immediate" spaced feedback 
of knowledge of correct response with those groups receiving the feed- 
back delayed until the end of the film in a massed presentation. lEhis 
comparison is shown in the following design matrix: 





RESPONSE 


FWBkCK. 


Overt 


Coveirb 


AV Immediate 


1 


2 


AV Delayed 


7 


8 



TABLE 6 



IMMEDIATE FSKDBACK VS RESPONSE (ANALYSIS OP COVARIANCE) 
( Covariates: Non-Language IQ and Vocabulary) 









Criterion 


Coviarate X 


Adjusted 

Criterion 

X 






N 


X 


cr 


Non-Lang. 

IQ 


Vo cab. 


1 . 


AV Immediate 
(Overt) 


54 


42.91 


12.90 


105.03 


68.55 


1+3.79 


2. 


AV Immediate 
(Covert) 


34 


42.27 


8.39 


106.35 


67.65 


1 + 3 . 3 '^ 


5 . 


Audio Immediate 
(Overt) 


35 


45.63 


9.01 


106.03 


72.60 


1 + 2.28 


4 . 


Audio Immediate 
(Covert) 


36 


44.75 


10.08 


102.64 


71.78 


1+1+.22 





df 


SS 


MS 


F 


Prob. 


Immediate Feedback 


1 


3.27 


3.27 


0.09 


— 


Response 


1 


19.11 


19.11 


0.54 


— 


Interaction 


1 


49.81 


49.81 


1 . 4 l 




Within 


133 


4691.93 


35.28 
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The results of the analysis are ^ovn in Table 7* significant dif- 
ferences were found between the two active response modes, between the 
two feedback modes, or for the interactions among these variables under 
conditions of "immediate" or "delayed" feedback of knowledge of correct 
response with the correct answers shown. 

The final analysis of the response and feedback variables com- 
pared the performance of the Overt Response/lmmediate Audiovisual Feed- 
back (Film Treatment l) with the correct answer given group with the 
0ve3rb Response/immediate Audiovisual Feedback without the correct An- 
swer given group (Film Treatment 13) • These two treatments were iden- 
tical except that Treatment I 3 did not provide the explicit feedback of 
the correct answers; that is, it did not show the correct answers for 
each page of the workbook on the screen. Subjects presented with Treat- 
ment 1 saw the relevant portion of the film and also saw the correct 
answer for each page of the workbook projected on the screen. The re- 
sults of the one-way analysis of covariance are shown in Table 8. Ro 
significant difference in perfoimance was found between the two groups, 
thus indicating that the addition of the correct workbook answer on the 
screen in addition to the review sequence presented after each response 
was not a contributing factor to learning. 

In summary, the results on the six analyses of the response- 
feedback interactions revealed only two statistically significant com- 
parisons — both for the feedback variables. Audiovisual or audio feed- 
back of knowledge of correct response, either immediate or delayed, 
were found to be superior to no feedback at all. Althou^ the differ- 
ences were not statistically significant, there appeared to be a sli^t 
advantage for "immediate" audiovisual feedback of knowledge of correct 
response over "delayed" feedback massed at the end of the film under 
both overt and covert response conditions. There was no apparent ad- 
vantage demonstrated for projecting correct workbook answers on the 
screen in addition to presenting a review sequence that served as feed- 
back to the workbook response Just made. Neither overt responses nor 
coveiti responses were found to be superior to the other under any of 
the feedback conditions. Finally, the Expository No Response treat- 
ment, in which no active responses were made nor feedback given, was 
as effective as any of the other experimental treatments. 



Review Modes 



A single one-way analysis of covariance was performed to deter- 
mine the relative effects of the three modes of using review materials 
in the presentations. The review sequences were identical to the feed- 
back sequences (without correct answer being shown), but performed a 
review function rather than a feedback function because they were made 
without student response. Comparisons were made among Spaced Audio- 
visual Review, Spaced Audio Review, and Delayed Massed Audiovisual Re- 
view (at end of film). The Expository No Response treatment was added 
to the analysis as a means of comparing the review variable with the 
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TABLE 7 



IMMEDIATE MD DELAYED FEEDBACK AJUD FESPONSE 
(AMLYSIS OF COVARIANCE) 
(Covariates: Non-Language IQ and Vocabulary) 









Criterion 


Co variate X 








N 


X 


O’ 


Non-Lang. 

IQ 


Vo cab . 


Criterion 

X 


1. 


AV Immediate 
(Overt) 




42.91 


12.90 


105.03 


68.55 


43.46 


2 . 


AV Immediate 
(Covert ) 




42.27 


8.39 


106.35 


67.65 


43.08 


7 . 


AV Delayed 
(Overt) 


29 


44.14 


9.93 


117.79 


72.69 


41.64 


8. 


AV Delayed 
(Covert) 


33 


40.76 


10.80 


105.67 


67.79 


41.54 



‘ 


df 


SS 


MS 


F 


Prob. 


Feedback 


1 


88.07 


88.07 


1.80 


— 


Response 


1 


1.72 


1.72 


0.04 




Interaction 


1 


.60 


.60 


0.01 




Within 


124 


6079.13 


49.02 







TABLE' 8 



FEEDBACK WITH AND WITHOUT COEEECT ANSWERS SHOWN 
(ANALYSIS OF COVARIANCE) 

(Covariates: Non-Language IQ and Vocabulary) 







Criterion 


Covariate X 


Adjusted 

Criterion 

X 




N 


X 


(J 


Non-Lang. 

IQ Vocab . 


1, Feedback With 
Answer 




42.91 


12.90 


105.03 68.35 


45.19 


lj>. Feedback Without 
Answer 


35 


46.86 


9.81 


112.11 7^.89 


44.65 





■ df 


SS 


MS 


F 


Prob. 


Feedback With and 
Without Answer 


1 


4.81 


4.81 


OolO 


' 


Within 


65 


3083.63 


47.44 







conventional flM treatment. The results of the analysis are given in 
Table 9 » 

Although the dliferences among the four treatment versions were 
not statistically significant, the Massed Audiovisual Review mode tos 
the least effective, and the basic Expository Ho Response film showed 
a small superiority over all review modes. 

As was the case with the active response and feedback variables, 
the review variable had no effect upon increasing learning from tte ex- 
perimental film. However, there again appeared to be a slight adva - 
tage to spacing the review rather than massing it at the end of the 

film. 



Film vs No Film 

A final one-vay analysis of covariance was performed to deter- 
mine whether or not learning had resulted from exposure to the experi- 
mental films. The test perforraance of the Expository No Response group 
wL compared with that of the Control group that took the perfo^nce 
test but did not see any of the films. The results of this analysis are 

given in Table 10. 
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TABLE 9 



REVIEW AND NO REVIEW (ANALYSIS OF COVARIANCE) 
(Covariates: Non-Language IQ and Vocabulary) 







Criterion 


Covariate X 


Adjusted 

Criterion 

X 




N 


X 


a 


Non-Lang. 

IQ 


Vo cab. 


9 . E 3 q)Ository 

No Response 


50 


45.57 


11.17 


104.45 


72.15 


41.50 


10 . Audiovisual 


52 


42.00 


9.9^ 


101.47 


72.22 


40.50 


11 . Audio Only 


55 


38.97 


10.82 


102.51 


66.54 


40.10 


12 . Massed Audio- 
visual 


50 


35.80 


15.05 


104.95 


65.07 


58.56 





df 


SS 


MS 


F 


Prob. 


Review vs 
No Review 


5 


158.48 


46.16 


0.95 


— — • 


Within 


121 


5869.41 


48.51 







TABLE 10 



FIIM AND NO FILM (ANALYSIS OF COVARIANCE) 
(Covariates: Non-Language IQ and Vocabulary) 







Criterion 


Covariate X 


Adjusted 

Criterion 

X 




N 


X 


a 


Non-Lang. 

IQ 


Vo cab . 


9. Expository 

No Response 


50 


43.57 


11.17 


104.45 


72.13 


. 42.65 


l 4 . Control Group 


55 


26.64 


7.92 


106.50 


66.64 


27.82 





df 


SS 


MS 


F 


Prob. 


Film and 
No Film 


1 


5194.01 


5194.01 


71.95 


< .001 


Within 


59 


2619.77 


44 . 4 o 
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The film group showed a superior performance over the Control 
group significant at greater than the .001 level. Thus_, it could he 
concluded that the experimental films were making a significant contri- 
bution to the learning of the content. 



Analysis hy Learner Characteristics 

Analyses were made of two learner characteristics in order to 
determine whether or not there was a learning differential in the way 
different types of learners responded to the different experimental 
treatments. The learner characteristics selected for analysis were the 
two covariates; Non-Language IQ and Vocabulary ability. The experimen- 
tal population was split into Hi^ IQ (IQ ^ lOl) and Low IQ (IQ ^ lOO) 
and into Hi^ Vocabulary (above the mean) and Low Vocabulary (below the 
mean). The same comparisons made in the main analysis were made for 
each of these split groups, with Vocabulary achievement used as the co- 
variate with the IQ analyses and Non-Language IQ used as the covariate 
with the Vocabulary analyses. The unadjusted mean scores and standard 
deviations for these split groups are given in Table 11. 



Non-Language IQ 



The results of the one-way and two-way analyses of covariance 
comparing post-test performance for the Hi^ and Low IQ groups are sum- 
marized in Table 12. The only statistically significant finding was 
for the expected superiority of the Expository No-Response film over the 
Control group, but this difference was found for both the Hi^ and Low 
IQ groups. 

It may be concluded that none of the experimental treatment con- 
ditions interacted differentially with the IQ of the learners. 



Vocabulary Achievement 

The results of the one-way and two-way analyses of covariance 
comparing post-test performance for the Hi^ and Low Vocabulaiy achieve- 
ment groups are summarized in Table 13 . As with the total groups and 
the Hi^-Low IQ groups, the High and Low Vocabulaiy achievement groups 
receiving the Expository No-Response film treatment demonstrated signi- 
ficantly superior performance to the Control groups. In addition, sev- 
eral other significant differences between Hi^ and Low Vocabulary 
groups were obtained. In the Hi^ Vocabulary group, but not in the Low 
group, the overt response mode interacted significantly with the audio- 
visual feedback mode and the covert response mode' with the audio feed- 
back mode to increase the learning. The Low Vocabulary group appeared 
to profit from either audio or dela;^ed feedback after making overt re- 
sponses. Having no feedback after a covert response was a definite 
handicap to learning for this group as was the receiving of review in- 
formation at the end of the film rather than spaced throu^out. For 
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TABLE 12 



RESULTS OF ANALYSIS OF COVARIANCE 
HIGH AND LOW NON-LANGUAGE IQ 
(Covariate: Vocabulary) 





Hi^ IQ 


Low IQ 


ANALYSES 


df 


F Prob . 


df 


F 


Prob. 


Response vs Feedback 
(Ix3x5x7) X (2x4x6x8) 

Response 

Feedback 

Interaction 


l/l4o 

5 /i 4 o 

5 /i 4 o 


OHO 

• • • 

o\ o 

1 1 1 
1 1 1 


1/105 

3/103 

3/103 


0.48 

0.71 

1.26 


— 


Overt vs No-Response 
(Ix5x5x7x9) 


4/86 


0.95 


V61 


0.71 


cS>' 


Covert vs No-Response 
(2x4x6x8x9) 


4/81 


o.6o — 


4/69 


1.15 


— 


Immediate Feedback vs Response 
(1x5) X (2x4) 

Response 

Immediate Feedback 
Interaction 


1/75 

1/75 

1/75 


0.00 

0.05 

0.98 


1/54 

1/54 

l/5‘^ 


1.11 

1.06 

0.05 


— — 


Immediate and Delayed Feedback 
vs Response (1x7) ^ (2x8) 

Response 

Feedback 

Interaction 


1/68 

1/68 

1/68 


0.01 

1.51 — 

0.58 


1/52 

1/52 

1/52 


0.77 

0.01 

5.28 


< .10 


Feedback With and Without 
Correct Answers (1x15) 


5/67 


0.06 


1/27 


0.12 


— 


Review and No Review 
(9x10x11x12) 


1/36 


0.67 


5/50 


0.50 


— 


Film and No Film (9xl4) 


1/52 


54.02 < .001 


, 1/25 


20.86 


< .001 
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TABLE 15 



RESULTS OF ANALYSIS OF COVARIANCE 
HIGH AND LOW VOCABULARY ACHIEVEMENT 
(Covariate: Non-Language IQ) 





Hi^ IQ 






Low IQ 




ANALYSES 


df 


F 


Prob. 


df 


F 


Prob. 


Response vs Feedback 
(1x5x5xT) X (2 x 4 x 6 x 8) 

Response 


1/123 


0.86 




1/120 


0.19 




Feedback 


3/123 


1.64 


— 


3/120 


4.19 


< .01 


Interaction 


3/123 


1.97 




5/120 


2.51 


< .10 


Overt vs No-Response 


V 73 


2.04 




k/ 7 k 


0.95 


— 


(Ix 5 x 5 x 7 x 9 ) 

Covert vs No-Response 


V 73 


1.55 




VT 7 


5.11 


< .01 


( 2 x^x 6 x 8 x 9 ) 

Immediate Feedback vs Response 
( 1 x 5 ) X ( 2 x 4 ) 

Response 


1/67 


0.87 




1/62 


1.24 




Immediate Feedback 


1/67 


0.05 




1/62 


2.71 




Interaction 


1/67 


8.19 


< .01 


1/62 


0.59 




Immediate and Delayed Feedback 
vs Response ( 1 x 7 ) x ( 2 x 8 ) 

Response 


1/60 






1/60 


0.74 




Feedback 


1/60 


2.56 


— 


1/60 


0.50 




Interaction 


1/60 


0.54 




1/60 


1.05 




Feedback With and Without 


1/29 


0.08 




l/ 3 ‘^ 


505 


'< .05 


Correct Answers ( 1 x 15 ) 
Review and No Review 


3/51 


1.12 




3/66 


10.54 


< .001 


( 9 x 10 x 11 x 12 ) 

Film and No Fi 3 m ( 9 xl 4 ) 


1/29 


82.84 


< .001 


. 1/28 


29.70 


< .001 



this group, review by sound only was less effective than review by the 
audiovisual mode, and they found that their performance was increased 
if they did not see the correct answer projected on the screen after 
the feedback sequence. 

It may be concluded that the Low Vocabulary group reacted more 
differentially to the stimulus modes than did the Hi^ Vocabulary 
group. Its performance was adversely affected by receiving no feedback 
of knowledge of correct response after making covert responses and by 
audio review only. However, it gained from audio or delayed feedback 
and from the omission of the correct answer for comparing their re- 
sponses after audiovisual feedback. 



Analysis of Performance Subtests 

The performance post-test consisted of items that tested Know- 
ledge, Comprehension, or Application as defined by Bloom (1956). Al- 
though these items were distributed throu^out the test, when separately 
analyzed they comprised the subtests, unadjusted scores of which are 
reported in Table 2 for each treatment group. 

The pattern of perfoimance on the three subtests was similar to 
that on the total performance test. These results are presented in 
Table ik. Only on the Application subtest were there significant dif- 
ferences in performance, and these tended to be in the same direction 
as on the other subtests. The no feedback condition was found to be 
significantly less effective than immediate feedback of knowledge of 
correct response. Audiovisual immediate feedback interacted positively 
with overt response and audio immediate and massed delayed feedback 
with covert response. As with the other analyses, the Expository No- 
Response film treatment was significantly more effective than no film 
at all, except that, in the case of the Application subtest items, 
this difference was significant at only the .05 level, suggesting that 
the Application subtest items depended less on the film presentations 
for information than did the Knowledge and Comprehension subtest items. 

In summaiy, there appeared to be little difference in the pat- 
term of performance among the three subtests or the total test, with 
the exception of some differences for the Application subtest. 



Analysis of Subtest Items Practiced and 
Not Practiced in the Workbook 



The performance post— test consisted of the 19 items that were 
specifically practiced in the workbook by those treatment groups making 
overt or covert participative responses and of the 48 items that were 
not specifically practiced. Althou^ these items were distributed 
throu^out the test, when separately analyzed they comprised the two 
svbtests. 
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TABLE Ik 



EESULTS OF ANALYSIS OF COVARIANCE SUBTESTS: 
KNOWLEDGE, COMPREHENSION, APPLICATION 
(Co variates: Non-Language IQ and Vocabulaiy) 







Knowledge 


Con5)rehen . 


Applicat . 


ANALYSES 


df 


F 


Prob. 


F 


Prob. 


F Prob . 


Response vs Feedback 
( 1 x 5 x 5 x 7 ) X ( 2 x 4 x 6 x 8 ) 

Response 


1/251 


0.02 




0.94 




0.29 ~ 


Feedback 


5/251 


1.87 


— 


1.91 


— 


5.47 <.C 5 


Interaction 


3/251 


0.65 


mmmm 


1.50 


' 


2.26 <.10 


Oveiii vs No-Response , 


V152 


0.50 




0.44 




2.29 <.10 


(Ix 5 x 5 x 7 x 9 ) 

Covert vs No-Response 


V155 


1.99 




1.82 




2.61 <.05 


( 2 x 4 x 6 x 8 x 9 ) 

Immediate Feedback vs Response 
( 1 x 5 ) X ( 2 x 4 ) 

Response 


i/;i 33 


1.45 




0.06 




0.01 — 


Immediate Feedback 


1/133 


0.84 





0.10 


— - 


0.08 — 


Interaction 


1/133 


0.01 




0.92 


• ' 


6.48 <.05 


Immediate and Delayed Feedback 
vs Response ( 1 x 7 ) x ( 2 x 8 ) 

Response 


1/124 


0.06 




0.51 




0.22 — 


Feedback 


1/124 


0.51 


— 


2 . 4 o 


— 


2.71 ~ 


Interaction 


1/124 


0.66 




0.00 


mmmm 


4.26 <.05 


Feedback With and Without 


1/65 


0.15 




1.98 




0.00 — 


Correct Answers ( 1 x 15 ) 
Review and No Review 


3/121 


0.57 




o „68 




0.81 — 


( 9 x 10 x 11 x 12 ) 

Film and No Film ( 9 xl 4 ) 


1/59 


124.40 


<.001 


0 

• 

0 

0 

A 

• 

0 

0 

H 


5 . 4 i <.05 
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The results, presented in Table 15, show only one significant 
difference in perform^ce — that for the interaction between immediate 
audiovisual and audio feedback and overt or covert response for the un- 
practiced items. This advantage was to the treatment group that com- 
bined covert response with audio feedback. Althou^ not significant, 
the practiced items, in this comparison, were most effectively learned 
when overt response was combined with audio feedback. With this one 
exception, all treatment groups appeared to perfoim as well on the non- 
practiced items as on the practiced itenis. 

A separate analysis , presented in Table l6, compared the per- 
formance on the subtest items practiced and not practiced in the work-r 
book in terms of percent answered correctly for the nine workbook 
groups and the four non-workbook groups. Although the workbook groups 
answered 12. more of the practiced items than the non-practiced items, 
the non— workbook groups (which had no workbook practice on any of the 
items) also answered 10. 6^ more of the practiced than the non-practiced 
items. This represented an insignificant difference of only 1.8^ items 
correctly answered between the practice and non-practice groups. There 
appeared, however, to be a sm a ll overall advantage for the Workbook 
groups over the Non-Workbook groups~5.^^ on the Practiced items and 
5 . 6 ^ on the Non-Practiced items. 

It may be concluded that the test items previously practiced in 
the workbook, correct answers to which were then fed back to the learn- 
er, were not learned appreciably better than those test items that re- 
received no workbook practice. 



Qualitative Analyses 

Measures were made of the state of mind of the subjects as they 
experienced the experimental situation. The semantic differential 
scale, a student reaction scale, and open-ended questions were used to 
probe feelings about the stimulus. 



Sp^mantic Differential 

The semantic differential Instrument (see Appendix D), consist- 
ing of eleven 7-point scales, was administered about two weeks follow- 
ing the administration of the stimulus film treatments. The concept, 
"The Story You Saw About the Eco-System, " was rated on each of the 
scales. Since the purpose of the analysis was comparison among treat- 
groups, t— tests were computed to establish the order of differ- 
ences between means for each group. Differences fo'und to be signifi- 
cant at the .05 level or less are presented in Table I 7 . The means 
for the groups and items referenced are presented in Table I 8 . The 
tables may be used together. Por example, it was found that treat- 
ment groups 1 and 2 differed significantly at the .05 level on item 7 
of the scale (KEPETITIVE/VARIED) . Their means were 4.^4 and 5.45; that 
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TABIE 15 



EESULTS OF MALYSIS OF COVARIANCE SUBTEST ITEMS PRACTICED 
AND NOT PRACTICED IN THE WORKBOOK 
(Covariates: Non-Language IQ and Vocabulary) 







Practiced 

(19 items) 




Not Practiced 

(48 items) 


ANALYSES 


df 


F Prob . 


df 


F Prob . 


Response vs Feedback 
( 1 x 5 x 5 x 7 ) X ( 2 x 4 x 6 x 8 ) 










Response 

Feedback 

Interaction 


1/224 

5/224 

1/224 


0.81 — 
1.54 — 
1.57 — 


1/224 

5/224 

1/224 


0.21 — 
1.97 — 
1.57 — 


Overt vs No-Response 
(Ix 3 x 5 x 7 x 9 ) 


4 /l 4 o 


1.07 — 


4/l4o 


0.96 — 


Covert vs No-Response 
( 2 x 4 x 6 x 8 x 9 ) 


4/156 


2.27 < .10 


4/156 


1.88 — 


Immediate Feedback vs Response 
( 1 x 5 ) X ( 2 x 4 ) 










Response 

Immediate Feedback 
Interaction 


1/120 

1/120 

1/120 


0 .l 4 — 
0.58 — 
1.65 — 


1/120 

1/120 

1/120 


1.51 — 
0.00 — 
4.46 < .05 


Immediate and Delayed Feedback 
vs Response ( 1 x 7 ) x ( 2 x 8 ) 










Response 

Feedback 

Interaction 


1/110 

1/110 

i/no 


0.25 — 
0.05 — 
0.94 — 


1/110 

1/110 

i/no 


0.11 — 
0.51 ~ 

0.07 — 


Feedback With and Without 
Correct Answers ( 1 x 15 ) 


1/62 


0.21 — 


1/62 


0.04 — 


Review and No Review 
( 9 x 10 x 11 x 12 ) 


5/106 


0.29 — 


5/106 


0.92 — 
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TABLE 16 



PERCENT OF PRACTICED AND NON-PRACTICED SUBTEST ITEMS ANSWERED CORRECTLY 

•RY WORKBOOK AND NON-WORKBOOK GROUPS 





Practiced 

Items 

i 


Non-Practiced 

Items 

i 


Difference 


Workbook Groups 


73.1 


60.7 


12.4 


Non-Workbook Groups 
(9,10,11,12) 


67.7 


57.1 


10.6 


Difference 


5 .^ 


3.6 


1.8 
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^irst figure presented is the item number; second is level of significance determined by t-^test. Read group to l e ft 
first, then group on top. If sign for the item number is negative, read top group first. 
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2.69 3.10 5.1^ ^.38 3-31 2.69 3**^5 3.72 3*86 2.97 ^-00 
2.58 2.82 4.94 4.00 3.21 3.00 3.88 4.1S 3.72 2.94 3.75 
2.56 3.00 4.81 4.44 5.03 2.97 4.06 4.28 3-72 3-25 4.16 
2.50 2.64 4.97 4.63 3'31 2.72 4.11 4.42 3*87 3*47 4.00 



is. group 1 found the stimulus films significantly more varied than did 
group 2. Inasmuch as the scalar center is h, group 2 thou^t the film 
was sli^tly repetitious, and group 1 found it slightly varied. 



In general, all of the film treatments were perceived to he 
GOOD, WORK, KIND, SLOW, STRONG, PLEASANT, MASCULINE, and lOUD. The 
largest number of significant differences between treatment groups 
(thirteen differences) was for the FAST/SK)W dimension of the 
with the Overt Response/Audio Feedback group (#3) finding the film o 
be significantly "slower" than ei^t of the remaining twelve treatments 
On the other hand, the Audio Review group (#ll) perceived their film to 
be significantly "faster" than four of the other groups. The Audio- 
visual Review group (#10) found their film to be more feminine 
five of the other groups, the only treatment group that interacte s g- 
nificantly on the FEMININE/MASCULINE dimension of the scale. 



The two treatment groups utilizing overt response— Film ^ea^- 
ment 1 with audiovisual feedback and Film Treatment 3 with audio feed- 
back-each showed ten significant differences with other treatmen^, 
the largest number for any of the film treatment groups. The Overt Re- 
sponse Audiovisual/Feedback group (#l) perceived their film to “be more 
"varied" than three of the other groups and lighter t^n four of the 
other groups. The Overt Response /Audio Feedback group (#3; perceived 
their film to be significantly "slower" than ei^t of the other treat- 
ments. 



Student Reaction Scale 

A student reaction scale, consisting of five 5-point scales 
(see Appendl,x D), was administered at the same time as the semantic dif- 
ferential scale about two weeks following the administration of the 
stimulus film treatments. The purpose of the scale was to elicit rat- 
ings relative to the subjects' perceptions of the length of the film, 
liking for the film, learning from the film, test difficulty, and 
helpfulness of the workbook in learning the material. Since the pur- 
pose of the analysis was comparison among treatment groups, t-tests 
were computed to establish the order of differences between means for 
each group. Differences found to be significant at the .05 level or 
less are presented in Table 19- The means for the groups and the ques- 
tions asked are presented in Table 20. The tables may be used together. 

In general, there was a tendency for all the treatment groups 
to rank the film treatments close to the mean. They considered that 
the film treatments were a little longj they liked them somewhat; they 
thought they learned something from them; they thought the test was 
fairly easy; and they felt the workbook helped them somewhat in learn- 
ing the material. 

The largest number of significant differences between treatment 
groups was for the Overt Response/Audiovisual Feedback group (fl), this 
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SIGNIFICANCE OF DIFFEEENCES BETWEEN GEDUP MEANS ON QUESTIONS IN STUDENT EEACTION SCALES 
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2. Covert /ay Feedback 


3 . Overt/Audio Feedback 


4. Covert/Audio Feedback 
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7 . Overt /Delayed Feedback 


8. Covert/Delayed Feedback 


9 . Expository No Response 


10. AV Review 


11. Audio Review 


12. Delayed Masses Review 


13 . Overt /ay Feedback 
Without Correction 
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■’‘First figxire presented is the item niunber; second is level of significance determined by t-test. Bead group to left first 
then group on top. If sign for the item number is negative, read top group first. 
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group considering the film significantly "shorter" than ei^t of the 
other groups. The Audiovisual Review group (^lO) also considered the 
film "shorter" than seven of the other groups, hut the Audio Review 
group (^ll) perceived it as significantly "longer" than two of the 
other groups. The Delayed Massed Review group "liked the film 

significantly better than four cf the other groups, hut thou^t the 
test was "harder" than did three of the groups. The Overt Response/ 
Audiovisual Feedback group (#l) felt that the workbook was more "help- 
ful" in assisting them to learn. There were no significant differences 
between any of the groups in their assessment of their level of "learn- 
ing" from the film. 



Open-Ended Questions 

Three open-ended questions were asked on the same form contain- 
ing the student reaction scale (see Appendix d). These responses were 
obtained about two weeks following the administration of the stimulus 
film treatments. 

To the question, "Which part did you like best ?" most groups 
were about equal in their responses. The most frequently given answers 
related to the curious living habits of fish. 

To the question, "Which part did you like least ?" again the 
groups were about equal in their responses. There were frequent men- 
tions of "the workbook" and "stopping the film" in response to this 
question. A number of subjects also responded "what fish eat" or its 
equivalent, thus confusing the rather clear-cut preference for knowing 
about the curious habits of fish. 

To the question, "Did you have any trouble understanding it?" 
530 subjects stated they had no trouble imder standing, and 76 said 
they did have trouble. 



^5 



CHAPTER IV 



CONCLUSIONS, DISCUSSION, AND IMPLICATIONS 



This chapter will present the specific conclusions that may "be 
derived from the data, discuss the results of the study, and suggest 
implications of the study for the design of instructional media. 



Conclusions 



The following conclusions may be made from an analysis of the 
results of the study. 

A. Response and Feedback Variables : 

1. The feedback to learners of knowledge of results of the 
correctness of their responses to workbook questions answered during 
the showing of a general science film was found to significantly in- 
crease learning over a condition where no feedback was given. This 
superiority was found for knowledge of results given in either an 
audiovisual or an audio-only feedback mode and when given either imme- 
diately after the student response or when delayed until the end of the 
film. 



2. Wnen no feedback was given to learners regarding the cor- 
rectness of their responses under the "covert" response conditions, 
significantly inferior performance resulted. There were no significant 
differences in such learning, however, under the "overt" response con- 
ditions. 



5 . Neither "overt" responses nor "covert" responses were 
found to be superior to the other under any of the feedback conditions. 

4 . There appeared to be a slight, but statistically insignifi- 
cant, advantage for "immediate" audiovisual feedback of knowledge of 
results over "delayed" feedback massed at the end of the film under 
both response conditions. 

B. Review Variables ; 

1. There were no significant differences among the three modes 
of presenting the review sequences. However, the "massed" audiovisual 
review mode was the least effective, suggesting some slight advantage 
to spacing the review rather than massing it at the end of the ^ilm. 
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C. Conventional Expository Treatment: 



1. The Expository No Response film treatment, in which no ac- 
tive student responses were made, no feedback given, and which took 
one-half the time to present, was as effective as any of the other 
experimental treatments. 

2. The Expository No Response film treatment resulted in 
greatly superior performance over the Control group which saw none of 
the film treatments. Thus, it could be concluded that the experimental 
films were making a significant contribution to the learning of the 
content . 



D. Learner Characteristics ; 

1. Althou^ the higher mental ability students performed at a 
significantly higher level than the lower mental abili' y students, 
neither of these groups interacted differentially with the experimental 
variables studied. 

2. Those students with hi^er vocabulary achievement performed 
at a significantly higher level than those with lower vocabulary 
achievement, but the lower group interacted more differentially to the 
stimulus modes. The performance of the low group was adversely af- 
fected by receiving no feedback of correct knowledge of results after 
making covert responses and by the audio or delayed review. It profited 
from audio or delayed feedback after making overt responses and from 
the omission of the correct answer for comparing their responses after 
audiovisual feedback. 

E. Performance on Subtests ; 

1. The patterns of performance on the Knowledge and Comprehen- 
sion subtest items were similar to that on the total performance test. 
Howtver, on the Application subtest, the "no feedback" condition was 
found to be significantly less effective than "immediate feedback" of 
knowledge of results. Also "audiovisual immediate feedback" interacted 
positively with "overt response" and "audio immediate feedback" and 
"massed delayed feedback" with "covert" response. 

2. Test items previously practiced in the workbook, correct 
answers to which were then fed back to the learner, were not learned 
appreciably better than those test items that received no workbook 
practice. 

F. Qualitative Reactions ; 

1. On the semantic differential scale all of the treatment 
group generally positive reactions to material presented. No out- 

standing q.ualitative advantage could be discerned for any particular 
eiq)erlmental treatment . 



2. On a student reaction scale there was a general tendency 
for all the treatment groups to rank the experimental treatments close 
to the mean. The students considered the films to be a little long; 
but they liked them somewhat, thou^t they learned something from them, 
thought the performance test was fairly easy, and felt the workbook 
helped them somewhat in learning the material. 

3. The open-ended questions revealed no useful information 
from student opinions. 



Discussion 



Several results of the study warrant a more detailed discus- 
sion. In this section, those specific outcomes that appear to have 
implications for the design of instructional materials will be examined. 



General Effects of the 
Experimental Variables 

Contrary to expectations, the inclusion of various procedures 
for eliciting student response, providing feedback of knowledge of the 
correctness of these responses, or presenting review sequences in an 
instructional general science film apper .■’ed to have minimal measurable 
effects upon the learning of the content. The film version (Expository 
No Response treatment), produced in the conventional way without these 
design elements, resulted in as much learning and in about half the 
time. Previous research with response, feedback, and review variables 
(Allen, i960; Lumsdaine, I963) strongly indicated that these techniques 
should have resulted in Improved performance. Nhy did the results of 
this study contradict the earlier findings? There are four possible 
explanations . 

Pirst, the design variables (response, feedback, review), as 
employed in this experiment, may, in fact, have exerted no measurable 
influence on the learning of the content. The nature of the content 
may have been such that it did not interact with the variables studied. 
In the light of previous research evidence, this explanation is diffi- 
cult to accept. 

Second, th<=. performance test itself may have been insensitive 
to the variations in learning produced by the experimental treatments. 
There is some evidence that the experimental subjects found the test to 
be easier than the tryout population. Item analyses after the first 
tryout showed most items to be too difficult, and corrections were made 
to make them easier. The second tryout confirmed the results of these 
changes, the group mean being about 50^ correct answers. However, the 
mean for the entire experimental poTulation was about 65^ correct an- 
swers, this group finding the test much easier. Since the subjects 
from both the tryout and experimental schools were comparable on meas- 
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ured dimensions, the only change that might account for so large a 
shift was procedural. During the tryouts, rather complex IBM sense- 
score sheets were used, necessitating the transfer of answers from the 
test booklet to the answer sheets. Because the tryout subjects found 
these sheets difficult and hard to use, a procedure of direct answering 
in the test booklet itself was u:ilized with the experimental popule- 
tion. Thus, the procedure used with the tryout groups may have made 
the test appear to be more difficult. So the performance test may have 
been a sli^tly blunted instilment with a ceiling lower than desirable. 
However, it is felt that this slight insensitivity in the test was not 
sufficient to account for the failure of the experimental variables to 
produce increased learning. 

Third, the design variables built into the films may have re- 
sulted in stimuli that were too long, tedious, and monotonous. During 
the workbook exercises specified lengths of time were given for the 
answering of the questions, and all groups conformed to this allocation 
as a control measure. This procedure served to slow down the presenta- 
tion of the films for the more able subjects and may have tempered their 
involvement in the film. That the subjects perceived the films as slow 
was confirmed in the qualitative measures. However, similar reactions 
were made by the non-workbook groups, and it is felt that the monotony 
factor could not account for the failure of the experimental variables 
to produce increased learning. 

Fourth, the design elements that were built into the films, 
other tvipn the experimental -variables, may have resulted in a level of 
learning beyond -vriiich differences were difficult to obtain or to de- 
tect. The high level of learning (65^ of the maximum possible) 
achieved by the conventional Expository No Response group tends to 
confirm this. Looking at the script for this film (Appendix A) it will 
be seen that considerable repetition and redundancy was built in. An 
effort was made by the researchers to design as effective a film as 
possible (excluding the experimental variables). This fact may have 
resulted in a condition where the response, feedback, and review vari- 
ables performed an additive function only. Because the basic film had 
raised the level of achievement about as high as could be expected, 
these additive features had little opportqtdty to increase the learn- 
ing. A post-experimental review of the Expository No Response script 
shows that the concepts and words tested in the 19 performance test 
questions piacticed in the workbook were repeated a total of 85 times. 
The concept that marine life has a particular place in which to live 
and particular needs (Test Questions #3, #l6, #66) occurred ik times 
either as statements of the concept or as repetitions of the word 
"needs." The needs themselves (warmth, li^t, safety, food), which 
were tested in five different test questions (#10, #62, #63, #64, #65 )^ 
appeared in the script in 20 separate repetitions. Other practiced 
items — "system," "eco-system, " "plankton" — each appeared in the script 
seven times. Only two of the practiced items appeared in the script 
three or fewer times. These repetitions— on precisely those items that 
the response, feedback and review variables were designed to influence— 



may have served to mask out the effects of the experimental variably. 
Repetition, as has been pointed out in research reviews by Allen (1900; 
and Lumsdaine { 196 ^), is an important factor in increasing learning. 

The experimenters, in attempting to design an effective basic , may 
have underestimated just how powerful this variable could be. There is 
no assurance, of course, that this was the cause of the non-signific^t 
differences obtained in the study, but it is the most probable one of 
the four possible explanations. 



Response and Feedback 

The overall lack of significant differences on learner response 
variables warrants little discussion other than the comments made in 
the preceding section. The fact that no differences in performance 
were found between the "overt" and "covert" response groups is not sur- 
prising in the light of the research evidence. 

The sli^t advantage accruing to the groups that used the work- 
books, on both the practiced and non-practiced items, suggests that the 
making of active responses has some positive effect upon learning. The 
research evidence would indicate, however, that the effects should have 
been far greater than they were. 

It did appear that "covert" response, coupled with the different 
feedback modes resulted in a more variable performance than did overt 
response. Receiving "no feedback" resulted in a decided disadvantage 
for the "covert" response group; and it is possible to suggest that 
when a response is made covertly, it has more need for reinforcement 
through feedback of the correctness of the response than when the re- 
sponse is made overtly. This supposition may be supported by the re- 
sults of the analyses of learner characteristics wherein the low vocabu- 
lary achievement group performed significantly lower when it responded 
covertly and received no feedback of knowledge of the correctness of 
that response. 

The only significant finding in the analysis of the effects of 
the feedback variable was in the general superiority of "immediate 
feedback over "no" feedback when the "covert" mode of student response 
was employed. On the separate subtest analyses, however, it was found 
that this difference held only for the Application subtest. Apparently, 
the provision of feedback was more important in the learning of material 
that required the application of learning than in the comprehension of 
the material or the recall of facts presented. Evidence from previous 
research indicated that the feedback variable should have had far 
greater effect upon performance than it showed in the study. 
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Review 



The failure of the review variable to increase the learning 
may have been a function of the powerful influence of the repetition 
built into the basic content of the films as discussed above. In fact, 
review itself is inherently a type of repetition, and if the repetition 
has already occurred at a level great enough to establish leaming of 
the concept taught, there should be little expectation of increasing 
learning throu^ additional repetition. 

The superiority of the "immediate and spaced" review over the 
"massed and delayed" review, although statistically insignificant, was 
supportive of learning research and contrary to the findings of the 
Millar and Levine study (1952), which found that massing of review mate- 
rial at the end of a science film was significantly more effective than 
spacing the review sequences throu^out the film. 



Learner Characteristics 

The tendency for the low achievers in vocabulary to have their 
performance adversely affected by receiving no feedback of the correct- 
ness of their responses after responding covertly seems to be logical. 
This group ml ght be expected to require more reinforcement of responses 
made less actively. However, the same expectation might also be ex- 
pected from the lower mental ability students, and this effect was not 
observed. 

It is difficult to account for the extremely low performance of 
the low achievers in vocabularjr in the Delayed Massed Review group, in 
which the lower mental ability students also showed the poorest per- 
formance. Nor is there any exj;)lanation for the high performance of the 
lower vocabulary group on the Overt Response/Audiovisual Feedback With- 
out Correction treatment (Treatment No. I5). It mi^t be expected that 
the additional practice (as provided in Treatment No. l) would benefit 
this lower ability group. 



Implications 

The implications of this study for the design of instructional 
media seem to be more negative than positive. That is, the. anticipated 
effects from the response, feedback, and review variables studied were 
not obtained. This does not mean, of course, that such variables are 
not important in the design of instructional media; but that, under 
the conditions prevailing in this experiment, they did not produce the 
expected effects. 

It may be, as discussed above, that the repetition and redun- 
dancy built into the films were sufficiently pervasive to override the 
effects of the experimental variables. If so, this fact Itself has 
implications for the design of instructional materials. However, the 
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comparative effects of the response/feedhack/review variables and the 
repet it ion/redundancy variables were not studied in this experiment and 
would need to be compared under controlled conditions In a future study. 

The suggestion that low achievers benefit more from feedback of 
knowledge of the correctness of their responses has some implications 
for instructional media design, but this study was unable to draw pre- 
cise conclusions as to exactly what design elements were appropriate 
for what learner groups. 
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APPENLIX A 



SCRIPTS OF ^EXPERIMENTAL TREATMENTS 



EXPLANATION; The script shown inraiediately below was for the most elab- 
orate treatment used in the study; the Overt Response with Immediate 
Audiovisual Feedback and Correct Answer (Film Treatment No. 1). The 
illustrations show the key points in the film. Those scene numbers 
marked with an asterisk (*) indicate the places v/here the other experi- 
mental treatments differ from Film No. 1. Following the script for 
Film No. 1^ each of the specific changes made in the other films will 

be detailed , 




1. A fish has an interesting way of 
moving up and down in the water. 



2. It does this by means of its swim 
bladder ... 



3. ...which, if filled with air, 

enables it to rise in the water . . . 
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k. if partly filled or empty, to 
sink in the water. 



* 5 . Please turn to page one in your book. 
Look carefully at the question there. 
Then answer: Is it True or False? When 
a fish empties the air from its swim 
bladder, it will go deeper into the 
water? Mark an X in your book beside 
the correct answer: True or False. 

(10 SECONDS NO SOUND). Now look at the 
screen and listen to the ri^t answer. 



*6. When a fish empties the air from its 
swim bladder . . . 



* 7 . ... it will go deeper into the water. 



*8. From the answer on the screen, correct 
page one in your book. The correct 
answer is True. 




BLANK SCREEN 




X True 
false 

- - 
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11. ... yet over 70?^ of its surface is 
covered by water . . . water foimd in 
the great oceans of the earth. 




12. In these oceans are many fomns of 
animal life ... and plant life ... 
animal and plant life that we call 
marine life because they live in . 
the ocean or sea. 



13. There is marine life at all depths 
of the sea, but each foim of marine 
life has a particular place it usually 
lives. Some forms live near the 
brightly lit top ... some deeper down 
in the dimly lit area . . . and others 
in the very deep dark ocean . . . even 
near the earth on the bottom of the 
sea. So there is marine life at every 
depth in the sea. 
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ik. Changes in the nature of the sea ... 
its winds ^ tides, saltiness and so on 
. . . cause different foims of marine 
life to live \nere they live. 




*15 . Turn to page tT^o in your book . . . 

Look carefully at the question there, 
then answer: Is it True or False? 

Each form of marine life has a particu- 
lar place in the sea where it usually 

lives. Mark an X in your book beside BLAM SCREEN 

the ri^t answer: True or False. 

(15 SECONDS NO sound). Now look at the 
screen and listen to the right answer. 



*16. Each form of marine life has a 

particular place where it usually 
lives. ^ 




*17. From the answer on the screen, correct 
page two of your book. The correct 
answer is True. 




18. Also important to where each form of 
marine life lives are its own special 
needs. One of these needs is food. 
Each form of marine life needs its 
own kind of food. 
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21. The great circle of marine life . . . 
one form feeding on another ... is 
often called a food-chain. VJhat we 
see at work here is a system by which 
every form of marine life finds the 
food which it needs. 



22. Food, then, is a need that makes each 
form of marine life live where it lives. 
Marine life has many other important 
needs: like keeping from getting cau^t 
and becoming food itself ... or like 
getting light enough to see both its 
food and. its enem.ies ... or like keep- 
ing warm enou^ while moving toward 
li^ts, food, or safety. Each form of 
marine life has many needs — all re- 
lated to one another, and all parts of 
one system. 



25. Scientists give this system a rather 
big name: they call it an ecological 
system. 
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2k. But scientists sometimes shorten such 
big names. Many of them call this 
[ system the eco—system of the sea. 

f 

i 




* 25 . Turn to page three in your book. 

Look carefully at the question there. 
Then fill in the right words where 
there are now blanks. (2 MINUTES 
NO sound). Now look at the screen 
and listen to the right answer. 



BLANK SCEEEN 



*26. In the eco-system of the sea, diff- 
erent forms of marine life live where 
they do because of their own needs: 
like safety, food, li^t, and warmth. 
The relationship to one another for 
such needs for life in the sea is ^ 
called a system. 




*27. From the answer on the screen, cor- 
rect page three in your book. The 
correct answers are: eco or ecologi- 
cal; life or animals; needs; food, 
light and warmth in any order; and 
system. 







k 
















* % food 


. iilht 


^•wormth' ^ 


A 


A* 








28. Each form of msr.-ine life has its own 
place to live^ s own way of fitting 
into the eco-system of the sea, its 
own needs. Because of such needs, 
each form of marine liie seems to 
have a fence around its usual place 
to live. 
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29. A quite Imaginary fence ... one -whicli 
marks the limits, let us say, to which 
a fish usually will go. By l^its we 
mean the edges of that area within 
i^ich a fish might find enough food, 
yet not too much danger . . . enough 
li^t or darkness, yet not too much 
warmth or cold . . . Not too much, noi 
yet too little, of whatever it needs. 



30. We give a name to what keeps marine 
life inside such imaginary fences: 
we call it the Law of the Limits. 



31. If we knew all there is to learn about 
the Law of the Limits, we would know 
just where to find each form of marine 
life in the eco-system of the sea. 



*32. Turn to page four in your hook. l£)ok 
carefully at the q.uestion there: What 
name do we give to wb.at keeps marine 
life inside imaginary fences in order 
to stay alive? Write your answer in 
your hook. (40 SECONDS NO SOUND) 

Now look at the screen and listen to 
the right answer. 



*33. The Law of the Limits is what we call 
what it is that keeps marine life in- 
side imaginary fences in order to stay 
alive . 
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*34. From the answer on the screen, correct 
page four in your hook. The correct 
answer is the Law of the Limits. 



35. We shall learn much more about the 
e co-system of the sea and its law of 
the limits if we think about how one 
need of marine life can relate to all 
others. 



36. Think about light I Most of the li^t 
in the sea comes from the sun. But 
sunlight reaches nowhere neaj* so far 
throu^ the water as it does throu^ 
the air. 



37* Sunli^t comes down in many colorful 
rays. But the warmer rays of the sun 
reach down only 25O feet into a sea 
which is up to seven miles deep. 
Deeper than these bright rays can 
reach, the cooler rays light and warm 
the sea for another few hundred feet. 
The rest of the sea is quibe cold and 
dark as we go down ... as if in a sub- 
marine . . . down to the floor of the 
sea. 

*38. Turn to page five in your book. Look 
carefully at the question there. Im- 
agine you are in a submarine 400 feet 
down into the sea. If you look out 
into the sea, what color would the 
water be? Mark an X in your book be- 
side A, B, or C. (20 SECONDS NO 
sound) Now look at the screen and 
listen to the right answer. 
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*59* At a depth of 400 feet in the sea 
that is, in the middle area . . . the 
water is colored a dimly lit blue • 




*k0. Prom the answer on the screen, correct 
page five in your book. The correct 
answer is B. 





k2> Many forms of marine animals are 

colored in such ways that they blend 
into their surroundings and are hard 
for their enemies to see or find. 
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44, ... and some resemible the places in 
which they hide. 



45 . Those in the deep, dark depths of the 
sea may be able to flash lights of 
their own, helping them to see or to 
scare their enemies. 



*46. Turn to page six in your book. Look 

carefully at the question there. List 
three different ways in which marine 
animals hide from their enemies. 

Write your answer in your book. 

(45 SECONDS NO SOUND) Now look at the 
screen and listen to the right answers 
from which you may have chosen three. 



* 47 . Some marine animals blend into light 
above, some into darkness below . . . 



*48. . . . some into their immediate sur- 
roundings . . . 




BLANK SCBEEN 
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* 50 . Some flash lights to see or to scare 
their enemies. 




*51. Now correct page six in your book. 

You may have the answer in any order. 

A correct answer would be any three of 
these five: Blend into li^t above, 
into darkness below, into surroundings, 
swim away, flash lights. Now look at 
the screen. 



Blend -into light above 
Blen’d into . darkness below, 
Blent/ into- surroundings. 
Swim away 
Flash lights 



52. In the eco-system of the sea, then, 
light is related to warmth and to 
safety. Also to food. 




55. Earlier we learned that plants start 
the food-chain in the eco-system of 
the sea. Let us learn now how plants 
themselves are fed. The sea, like the 
land, has within it the chemicals, or 
fertilizer, which plants use as food. 
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5 ^. The plants of the sea, often without 
roots, and floating on the waves, soak 
up these chemicals and make them into 
food . . . 




55 . . . o throu^ a process called photo- 
synthesis ... a process that works 
only when li^t is present. 




*56. Turn to page seven in your book. Look 
carefully at the question there. In 
which layer of the sea — top, middle, 
or bottom — do plants soak up chemicals 
during photosynthesis? Mark an X in 
your book in the ri^t layer. (25 
SECOm)S WO SOUWD) now look at the 
screen and listen to the ri^t answer. 



BLANK SCEEEN 



*57. Plants soak up chemicals during 
photosynthesis in the top layer. 



*58. From the answer on the screen, cor- 
rect page seven in your book. The 
correct answer is the top layer. 
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59 - Where light and the ri^t chemicals 
are plentiful, photosynthesis can 
happen in millions of tiny plants . . . 
plants too small for one to see with- 
out help . . . 




60. ... plants called plankton. 




6l. The same name — ^plankton — is given to 
millions of tiny shrimp-like animals 
which live with and feed on the tiny 
plankton plants. 




62 . The food chain in the eco-system 
begins with these tiny plankton: 
plankton plants and plankton animals. 



*63. Turn to page eight in your book. Look 
carefully at the list of facts there. 
These facts need to be put in the or- 
der of their happening. Please fill 
in the numbers, 1 to 6, beside the 
facts in the list to show them in the 
correct order of their happening. Do 
this in your book. (1 MIITUTE 30 SEC- 
ONDS NO sound) Now look at the screen 
and listen to the right answer. 




BLANK SCREEN 
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*64. There is a correct order in the food- 
chain by -which light reaches the 
plants . « . 




* 65 . ... plants soak up chemicals . . . 




*66. ... photosynthesis happens to feed 
the plants . . . 




* 67 . ... the plants, like plankton-plants, 
are eaten by plankton animals . . . 







I 



1 




*70. From the answer 
rect page ei^t 
correct list of 
order: 6 , 2 , 



on the screen, cor- 
in your hook. The 
numbers is in this 

3 ^ 1 . 




71. In this food-chain of the e co-system, 
we must look for plant life mainly 
near the top of the sea. 




72. Marine animals in this bristly lit 
top of the sea eat plants and each 
other. 




73* A little deeper in the sea, in the 
dimly lit middle part, different 
forms of marine animals also eat 
each other. 
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7 ^« In addition, they eat some of the 
dead plants and animals •which drift 
do-wn from hi^er up in the sea. 



75. But most of the dead plants and 

anim Is ... the garbage and debris of 
the sea . . . drift down throu^ the 
deep darkness below . . . 



76. ... there to be eaten by very strange 
forms of marine einimals, some of •which 
may also eat each other. 



77. Miles deep throu^ the sea is, •what 
happens at the bristly lit top is 
related to ... 



78. ... •what happens at the darkest 
point belo^w. 
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79* From the ocean floor scientists bring 
up ooze and clay which is as much as 
two miles thick at the bottom of the 
sea. In such mud they find signs of 
dead animals and plants . . . and of 
chemicals made at the. bottom of the 
sea by very tiny organisms. 




80. These tiny organisms are called 
bacteria . 




8l. The chemicals made by the bacteria 
dissolve in the dark ocean waters . . . 




82. 



*83. 



... then rise in the moving waters 
to the top of the sea where plants 
use them for food during photosyn- 
thesis. 



Turn to page nine in your book. Look 
carefully at the instructions there. 
Beside each listed kind of marine life, 
mark where it will be found: T for at 

the top, M for in the middle, or B for 
at the bottom of the sea. Do this in 
your book. (45 SECONDS NO SOUND) 

Now look at the screen and listen to 
the ri^t answers. 




BLANK SCREEN 



1 




i 
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*84. In the eco-system of the sea, we find 
bacteria making chemicals at the bot- 
tom of the sea . . . 





*86. ... fish which blend into blue 
backgrounds in the middle . . . 



* 87 . ... fish which have to make their 
own li^t at the bottom . • « 





*88. ... and live plankton animals at 
the top. 
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*89 • From the answer on the screen, correct 
page nine in your bock. The correct 
order should read down like this: 

B, T, M, B, T. 




90. We have seen that li^t is related to 
warmth, safety and food in the eco- 
system of the sea. 




91. But the story of light and food is 
not complete until we learn how the 
dissolved chemicals come up from the 
ocean floor to reach the li^t. 





92. The sea is always in motion. Warm 
water at the top keeps moving away. 
Cold water, full of dissolved chemi- 
cals, keeps coming up from below to be 
warmed. But this change would be very 
slow were not for the winds over the 
sea. These winds push the warm waters 
away from the hot eq.uator, making such 
waters flow away like rivers throu^ 
the waves and tides of the sea. 




95. And such currents ... as these moving 
waters are called . . . flow away from 
the hot equator toward the cold poles, 
north and south. There the currents 
grov cold, sink to the bottom of the 
sea, then flow back toward ohe equa- 
tor along the ocean floor . . . 
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9^. ... there to rise and replace the 

warm water which has moved away from 
the equator. It is, then, the great 
movements of currents^ with their up- 
ward motion of water from helow^ which 
bring up the chemicals from the ocean 
floor for use in photosynthesis. 




*95. Turn to page ten in your hook. Lis- 
ten carefully to the instructions 
there. On the map of the currents 
there, mark an X beside the letter 

which is next to the warm current. BLAMC SCREEN 

Do this in your book. (20 SECONDS 
NO sound) Now look at the screen 
and listen to the ri^t answer. 



*96. The currents flowing away from the 
equator toward the north or south 
pole are waim currents. 




*97. From the answer on the screen, 
correct page ten in your book. 

The correct answer is an X beside 
the letter A. 




98. While nothing can halt the movement 
of the sea, the upward flow of chem- 
icals can be stopped by changes in 
temperature at different depths of 
the sea. Temperatures at the top of 
the sea may change greatly from summer 
to winter, and from north to south. 
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99- But in the dark bottom part of the 
sea, the water is a3.ways cold, thou^ 
never quite freezing. 



100. Vlhere this very d,eep area begins, 

the temperature is always about 55° 
above zero, then drops only about 
two degrees in going down to the 
ocean floor. 



101, Though water heats and cools much 

more slowly than air, the temperature 
above the deep dark area may change 
a great deal. 



102. ¥hat scientists call a thermocline 

begins where the li^t ends and where 
the sudden change to cold commences. 
This thermocline is like a wall be- 
tween the cold waters in the deep sea 
and the wanner waters nearer the top. 



105 . But this thermocline, or sharp dif- 
ference in temperature, is greatest 
in summer when the sun warms the 
upper waters of the sea. 
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lok. But in winter . . . when the difference 
in temperature above and below the 
thermocline starts to disappear . . . 
then the thermocline itself is no 
longer much of a wall. 




105. So it is, that each winter, tons of 
chemicals, dissolved at the bottom 
of the sea . . . 





106. ... sweep up to the surface ... 




107. 



* 108 . 



. . . and are swept by winds and cur- 
rents out to distant parts of the 
sea. 



Turn to page eleven in your book. 

Look carefully at the instructions 
there. The two diagrams there are 
of the currents and thermocline in 
summer and winter. Below each dia- 
gram on page eleven, write in whether 
it belongs with summer or with winter. 
(25 SECONDS NO SOUND) Now look at the 
screen and listen to the right an— 




BLANK SCREEN 



swer. 
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*109. The circle of the currents stops at 
the thermocline on the way up in 
summer . . . 




*110. ... but it breaks throu^ to become 
complete each winter . 




*111. From the answer on the screen, cor- 
rect page eleven in your book. Cor- 
rectly, you should have written sum- 
mer under the diagram at the left, 
winter under the diagram at the ri^t. 





Covert Response with Immediate Audiovisual Feedback and Correct 
Answer (Film Treatment No . 2 ): 

Because the response was "covert" rather than "overt," directions 
during the workbook exercises were as follows: "Do not write the 
answer, but think what it should be." This occurred in the fol- 
lowing scenes: 5, 15, 25, 52, 38, ^6, 56, 63, 83, 95, and 108. 



o 
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Overt Response vltb. Immediate Audio-Only Feedback and Correct Ansver 
(Fiiljii Treatment No. ^): 

During the feedback portions of the workbook exercises, directions 
were given to "listen to" the right answer. The audio portion 
only of the feedback was played as the subjects listened. This 
occurred in the following scenes: 5, 6 and 7; 15 and l6j 25 and 26 j 
32 and 33j 38 and 39j 46, 47, 48, 49.» and 50; 56 and 57; 63^ 64, 

65, 66 , 67^ 68 , pnd 69; 83^ 84, 85, 86, and 88; 95 and 96; 
lOB, 109 e and 110.^ At those places where the correct answer from 
the w; .1 l.v'ok was projected on the screen in Film No. 1, the screen 
remp; 1 i -<lank ar-d the answer was given in the audio track only. 
This oeriiC’^ed in the foiJLowing scenes: 8, IJ, 27, 31, 4o, 51, 58, 

70, 89, 91.. 111. 

Covert; Response with ijcmediate Audio»-0nly Feedback and Correct Answer 
(Film Trea.tment No". 4) ; ” 

The "response" portion of the film was the same as Film Treatment 
No. 2, and the "feedback" portion of the film was the same as 
Film Treatment No. 3« 

Overt R esponse with No Feedback (Film Treatment No. 5): 

All. the feedback and correct answer sequences were deleted. 
Immediately after the correct answer was written in the workbook, 
the narrator said, "Now look At tJhe screen," and the next exposi- 
tory sequence followed. This occurred in the following scenes: 

5, 6, 7 and 8; 15, I6 and 17; 25, 26 and 27; 32, 33 and 3^; 38, 39 
and 40; 46, 47, 48, 49, 50 and 51; 56, 57 and 58; 63, 64, 65, 66, 
67, 68, 69 and 70; 83, 84, 85, 86, 87, 88 and 89; 95, 96 and 97; 
108, 109, no and ni. 

Coverb Response with No Feedback (Film Treatment No. 6): 

The "response" portion of the film was the same as Film Treatment 
No. 2, and the "feedback" portion of the film was the same as 
Film Treatment No . 5 • 

Overt Response with Delayed Audiovisual Feedback and Correct A nswer 
(Film Treatment No.^7)i 

After the response was written in the workbook, the narrator 
said, "Now look at the screen," and the next expository sequence 
fonowed. At the end of the entire film (after Scene in), there 
was a short blank leader during which the narrator said. Now 
look at the screen. Let us prepare now to correct the answers 
we gave earlier in this film. Turn back to page one in your book. 
During this film you have been asked eleven questions. Let us 
maVp sure of the correct answers." There then followed a presen- 
tation of the feedback and correction sequences, one after the 
other: 6, 7 and 8; I6 and 17; 26 and 27; 53 an^ 3^; 39 and 40; 



hi, 48, 49, 50 and 5I5 57 and 58; 64, 65, 66, 67, 68, 69 and 7O5 
84, 85, 86, 87, 88 and 89; 96 and 97; 109, HO and 111. 

Covert Response with Delayed Audio-Only Feedback vith Correct 
Ansver (Filin. Treatment No. 8); 

The "response" portion of the film was the same as Film Treatment 
No. 2, and the "feedback" portion of the film was the same as 
Film Treatment No. 3 except that the feedback sequences were 
massed at the end of the film as in Film Treatment No. 7* 

Expository No Response (Film Treatment No . 9) i 

Only the basic expository material in the film was shown. All the 
scenes marked with an asterisk (*) in the script of Film No. 1 
were deleted. 

Audiovisual Review (Film Treatment No. lO); 

Following each of the expository sequences in the film, the 
"feedback" material was presented, introduced by the statement 
in the narration: "We have learned, then, that.... Both the 

response material and the correction of workbook material, con- 
tained in the following scenes, were deleted; 5 and 8, 15 and I7, 
25 and 27, 32 and 34, 38 and 40, 46 and 5I, 56 and 28, 63 and 70, 
83 and 89, 95 and 97, IO8 and 111. 

Audio Review (Film Treatment No. ll) ; 

This followed the same pattern as Film Treatment No . 10, except 
that the "feedback" material was presented without visuals in 
the audio mode only. 

Massed Audiovisual Review (Film Treatment No. 12); 

The basic expository material in the film was shown exactly as 
in Film Treatment No. -9. However, at the end of the film, the 
narrator said, "Let us now review the main points we have learned 
about the eco-system of the sea." Then followed the "feedback 
material contained in scenes 6 and 7J I6; 26; 33 J 39 J 47, w, 49 
and 50; 57; 64, 65, 66, 67, 68 and 69; 84, 85, 86, 87 and 88; 

109 and 110. 

Overt Response with Immediate Audiovisual Feedback but Wit hout the 
Correct Answer (Film Treatment No. 13)i 

This was the same as Film Treatment No. 1, except that the cor- 
rect workbook answer was not shown. Thus, the following scenes 
were deleted; 8, 17^ 27, 34, 40, 51^ 58, 70, 89, 97> Hi* 
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APPENDIX B 



W 0 R KBOOK 



Name 



School 



PLEASE DO NOT OPEN 
UNTIL ASKED TO DO SO 



Page 1 

Is it true or false: 

When a fish empties the air from its swim bladder, it 
go deeper into the water • 

_TRUE 

FALSE 



Page 2 



Is it true or false? 

Each form of marine life has a particular place in the 
sea where it usually lives. 

_TRUE 

FALSE 



81 



o 



Page 3 



Fill in the ri^t words where now there are blanks . 

In the system of the t 

different forms of marine li'V® 

where they do because of their own J 

like safety, , j an<3. . 

The relationship to one another of such needs for 
life in the sea is called a ,• 



Page 4 

Wliat riamp do we give to what keeps marine life inside imaginary 
fences in order to stay alive? 



Page 5 

Imagine you are in a submarine 400 feet down into the sea. If you 
looked out into the sea, -what color would the water be? 



a. brightly lit green 

b. dimly lit blue 



Co very black 



82 



Page 6 



List three different ways in which marine animals hide from their 
enemies. 



Page 7 



In which layer of the sea— top, middle, or bottom — do plants soak 
up chemicals during photosynthesis? Mark an X in the ri^t layer. 







Top 




Middle 




Bottom 


4^- 
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Page 8 



These facts need to he put in the order of their happening. Please 
fill in the nxamhers, 1 to 6 , beside the facts in this list to show 
them in the correct order of their happening. 



large marine life eats small mEirine life 
plants soak up chemicals 
small marine life eats plankton-animals 
photosynthesis takes place 
plankton-animals eat plankton-plants 
light reaches plants 



Page 9 



Beside each listed kind of marine life> mark where it will be 
found: T for at the top, M for in the middle, and B for at the 

bottom of the sea. 



bacteria making chemicals 
fish which eat plaiAton-plants 
fish which blend into blue background 
fish which have to make their own li^t 
live plankton-animals 
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On the map of currents shown below, mark an X beside the letter 
which is next to the warm current. 
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The twj diagrams shovn telow are of the currents and thej»cline In 
summer and winter. Below each diagram, write in idiether it Belongs 

with summer or winter. 





APEEUDIX C 



EEEFOBMANCE TEST 



The correct answers are indicated in the Perfoimance Test hy underlining 
in the case of the multiple-choice items or hy the Insertion of all the 
acceptable answers for the constructed responses. 



The Knowledge Subtest items we re as follows ; 1, 2, 3 , 4, 6, 8 , 9 , 10 , 

12, l6, 19, 20, 21, 27, 2b, 51, 5^^ 36, W , ^5, ^7^ ^ 9 , 51, 
55, 58, 62, 65, 64, 65. 

The Comprehension Subtest items wer e as follows ; 7, 11, I8, 22, 24, 

25, 26, 52, 57, ^1, ^2,~45, 46, 50, 52, 55, 5^, 56, 57, 66. 

The Application Subtest items were as follows ; 5, 15, 15, 17, 25, 

29, 50, 55, 55, 59, 59, 00, bi, by. 

The items practiced in the Workbook wer e as follows ; 5, 5, 8, 10 , I6, 

24, 5^, 56, '4o, 44, 46, 49, 51, 52, 62, 65, 64, 65, 66. 



The it ems Hot Practiced in the Workbook were as follows ; 1, 2, 4, 6, 

'7, 9, 11, '12, 15, 1^, 15, 17, 1», 19, 20, 21, 22, 25, 25, 26, 27, 28, 
29, 50, 51, 52, 55, 55, 57, 58, 59, 4l, 42, 45, 45, 47, 48, 50, 55, 
54, 55, 56, 57, 58, 59, 60, 61, 67. 



1. The planet earth is covered with 



1) 

2) 65i 

m 



water. 



2. Where in the sea would you expect to find no life? 

1) wherever it is too dark 

2) only where the water is cold 
5) life is everywhere 

4; in places where it is very deep 



5. Fish live where they do in the sea because they must 

1) search for chemicals to make food 

2) meet their own special needs 
5; have li^t for photosynthesis 
4) find warm temperatures 



o 
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4 . Some forms of marine life eat other forms, which are in turn 
eaten by still other forms. We call this relationship 

1) a food chain 

2) a circular motion 

3) photosynthesis 

k) the Lew of the Limits 



5. If we were to choose a word to describe how our safety is related 
to li^t, and light to the warmth of the earth, and those to the 
food we eat, we mi^t call this pattern a 

1) limit 

2) chain 

3) system 

4) circle 



6 . Each form of marine life 

1) eats most plants and animals 

2) must be kept warm 

3) must have li^t to grow 

4 ) needs its own kind of fooc^ or chemicals 



7. As part of the food chain of the sea, a fish is apt to be 

1) disobeying the Law of the Limits 

2) trapped by fishermen 
3 j safe among plants 

4 ) eat e n by other fish 



8 . The pattern of needs that makes all animals and plants in our 
world live where they do lu called the ecO"-systemj 

ecological system j 
law of the limits 



9. 



In the food chain of the 



sea, plants become food for animal^; fish ; 

plankton animals j 
plankton fish 



87 



V 



10 



. It was said in the movie that animals and plants living in the 
sea have four basio needs, these are: 

1) light, chemicals, food, and photosynthesis 

2) safety, chemicals, food, and light 
5 ) chemicals, food, warmth., and safety 
4) warmth, lights safety, and food 

11. What would happen to marine animals if all marine plants were 
to die? ■ 

they would die; they would starve : all would dlej 
there would be ho marine animals left 



12. A limit is 

1 ) a law I 

2 ) the edge of an area 

3 ; a little bit of something 
4) the outside of a thing 



People don't live under water because of the fact that they have 
a need for air in order to live. This is an example of the Law 
of the Limits as it affects human beings. Give two of your own 
examples of the Law of the Limits at work in the world around us. 

15) 

Accept reasonable applications of the Law of Limits : 

14) we need water, warmth^ food, etc.j fish can't live 
on land: we can't live in space j people need food ; 
we need oxygen; we must be safe 



15 . Imagine that we have found a small, strange creature struggling 
in a shady pool of water along the shore. It seems quite at home 
under water. It looks like a fish, yet it has fur. When we pick 
it up eind look at it in the sunli^t, it stops struggling and 
seems comfortable out of the water. In accordance with the Law 
of the Limits, we might conclude that the creature is a 

1 ) strange kind of fish cau^t in a tidal pool 

2 ) very young sea bird 

3 ) land mammal lost along the shore 

4) sea mammal trapped in a tidal pool 
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16. 






When scientists say "eco-system^ " they mean 

1) fish and plants that have similar needs 

2) sounds that reflect in patterns 

3) only fish and plants that live in the sea 

4) all living things have related needs 



17. The Law of the Limits 

1) determines the position of plants and animals in the food chain 

2) applied to all living things 

5) determines the number of fish living in the sea 
4) applies only to marine life 



18. If the earth were surrounded by water instead of air^ we would 
expect li^t to reach us from the sun. 

1) a little more 

2) much less 

5) a little less 
4) much more 



19. In clear water, the warm rays of the sun penetrate about 
into the sea. 

1) 150 feet 

2) 850 feet 

3) 7 miles 

4) 230 feet 



20. Which color light ray penetrates deepest into the sea? 

blue; violet; blue-violet; dim bluej dark blue; 

dark purple j purple; ultra violet (has to be cold color) 



21. Most of the sea is 

1) dim 

2) dark 

5) very light 
4) ll^t 
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22. VJhich of the following best describes the way in which sunlight 
penetrates the sea? 

1) both warn rays and cool rays penetrate all the way to 
the bottom 

2) warm rays penetrate more deeply than cool rays 

3) warm rays may be found only near the surface 

only the cool rays penetrate all the way to the bottom 



23. If a fish has a silver-colored belly, 

1) enemies below it cannot see it clearly 

2) it will look like marine plants 

3) enemies above it cannot see it clearly 

4) it will stay in the sand on the bottom 



2 k. Animals in the sea have many ways to escape being eaten. One 
example would be that some 

1) eat plants 

2) are blind 

3) big fish eat only little fish 

4) are varied in color 

25. Suppose a certain kind of fish is found in only one part of the 
sea, what ml^t we conclude about its needs? 

needs are met the re j needs are found there ; its needs 
are in this one area j it needs what is there ; it must 
be there for its food, warmth, light and safety 



26. Some fish can luminesce, that is, can produce their own li^t. 
Those fish would probably be found 

1) where enemies are many 

2) near the bottom 

3) in the dim part of the sea 

4) under rocks 



27- Plants use to make their food. 

1) chemicals 

2) bacteria 

3) plankton 
k) fish 





90 



28. The process whereby plants produce sugar in the presence of 
needed chemicals^ water^ and li^t is called photo synthe sis . 



29. If you were trying to grow sea plants in a tank^ and you had 
correct amounts of li^t, chemicals^ fish^ and sea water at 
the ri^t temperature^ you would probably fail because 

1) photosynthesis would stop 

2) there would be no wind 

3) there would be no currents 

4; it has to be the right season 

30. In accordance with the Law of the Limits^ we will find that moss 
grows on the shady side of trees in our forests because 

1) the temperature is too low on the other side 

2) it needs less light than most plants 

3) there is more water there 

4) there is less wind 



31. The food chain in the eco-system of the sea begins 
with plants 

plankton plants 

32. In this profile of the sea^ which of the numbered areas will 
contain most plants. 



1 

2 



3 

k 




33. The most important difference between plants and animals is that 

1) plants make their own food 

2) animals can move about 

3) animals have legs 

4) plants have roots 
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34 . Tiny living organisms found near the surface of the sea are called 



1) plankton 

2) bacteria 

3) fish 

4 ) plants 



35 * There are fewer animals in the sandy desert than in many other 
places on earth because 

1) it is very hot in the daytime 
. 2) most animals do not like sand 

3) few plants grow there that can he eaten 

4 ) there is no place to hide 



36. The Imaginary fence that keeps marine plants and animals within 
their own special place is called the Law of Limits . 

37 * Plants of the sea and grass of the earth are both found near the 
surface because 

1 ) they both have roots 

2) they are safest there 

3) they both need chamicals 

4 ) they both need li^t to make food 

38. Plankton is 

1) fish 

2) plant only 

3) animal only 

4 ) both animal and plant 



39. We know from the movie that dead animals and plants sink in the 
sea and become food for the fish that live at the bottom. 
Briefly^ give an example of how idiat happens at the top of the 
sea is related to what happens at the bottom. 

food comes from the top ; food chain j chemicalB rise to 
the top j fish (or plants) that die at the top go to 
the bottom, are changed to chemicals whidi rise to the 
top ; plant food comes from the bottom 
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40. The tiny organisms that assist In the decay of dead plants and 
animals are found 

1) at the thermo dine In the sea 

2) at the bottom of the sea 

5) In the dimly lit area of the sea 
4) In parts of the sea with most sunlight 

41, We know that animals and plants live where they do "because of 
their different needsj therefore^ If they went somewhere else, 

1) they could not return 

2) they would have to change or die 
5} they would have no difficulties 

4) It would "be uncomfortable but safe 



h2. Even In Its darkest part, the sea Is 

1) In motion 

2) warm 

5) full of plants 
k) free of chemicals 

45. In the deepest parts of the sea, the water temperature 

1^ Is always low 
2) changes with the seasons 
Is hipest near the equator 
h) changes with the weather 



kk. Most of the warm water in the sea moved toward the 

1) thermocline 

2) shore 
5) poles 
4; limit 

45. Temperatures at different levels of the sea are 

1) very different 

2) about the same 

5) subtly different 
4) exactly the same 
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46 . Which of the following best describes the movement of the sea? 





1 ) cool water moves toward the poles and rises to be warmed 

2 ) the wind cools the sea, and cold water sinks 

3) waim water moves toward the poles ^ sinksj and moves back 
to be warmed 

4 ) the sea moves up and down with the winds and back and forth 
with the seasons 



47. The rivers of the sea are called « 

1) waves 

2) currents 

3) movements 

4 ) gulfs 

48 . The warm and cold air in this room are in motion. We have seen 
that warm and cold currents in the sea are also in motion. 
Briefly describe how the motion of the air and the sea are alike. 

warm air cools and is displaced ! they both move in a 
circle ; warm to top, cold to bottom ; cold at bottom 
and warm at top ; warm air rises and cool air drops ; 
they both move in currents; move in same direction 



49. The chemicals used by plants in photosynthesis come from the 

1) sun 

2) thermo dine 

3) land 

4 ) bottom of the sea 



50. The sea heats and cools the air. 

1) mucdi faster than 

2) faster than 

3) the same as 

4 ) slower than 



51. At certain times of the year, there is a greater difference in 
temperature between the lifted surface and the darker ’ depths 
of the sea than at other times of the year. The level at which 
the temperature of the sea changes abruptly is known as the 



1) Law of the Limits 

2) top 

3) seasonal ciirrent 

4 ) thermo dine 
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52, The difference "between the temperature at the surface and the 
temperature at the "bottom of the sea is: 



1^ greatest in Summer, least in Winter 

2) least in Fall, greatest in Spring 

3 ) least in Jummer, greatest in Winter 
k) greatest in Fall, least in Spring 



53 * Currents of air, or winds, are very important to life in the 
sea "because winds 

1) slow the movement of the sea 

2 ) help form rivers or currents in the sea 

3) move de"bris along the "bottom of the sea 

k) allow li^t to reach sea animals 

5 ^. The temperature near the surface of the sea is hi^er than it 
is deeper down "because 

l) the "bottom of the ocean is always cold 

2) li^t and heat go only so deep 

3) there is more movement near the surface 

k) currents carry warm water to the surface 

55 • l^he temperature near the "bottom of the sea is a"bout 

l) 53 

2J55 

3) 55 

4 ) 70 

56. Briefly tell how dead marine animals and plants are used in 
the food chain of the sea: 

decayed "by "bacteria, which produce diemicals, which 
feed plants ; are eaten "by other animals (fish) at the 
"bottom; they are changed into chem 3 .cals j " bacteria 
make chemicals; they are changed into food ; they are 
eaten as they sink; they are eaten at the "bottom 



57. Most animals and plants of the sea o"bey the Law of the Limits 
and therefore 

1) often move from the top to the "bottom of the sea 

2) hi"bernate for the season 

5) avoid dangerous changes in temperature 
4 ; stay on the "bottom of the sea 
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58. In the profile of the sea shown "below, one temperature is in 
error. CShoose the temperature that would correct this error. 

i 




1) 35 should he 38 

2) 55 should he 60 
5) 22 should he 35 
5-) 70 should he 75 



59. On land, waim-hlooded animals move freely from relatively low 

temperatures to relatively hi^ temperatures. Whales, porposes, 
and turtles are warm-hlooded animals, therefore we would expect 
them to 

ij he found only in the wannest parts of the sea 

2) remain near the land on the floor of the sea 

3) stay as close to waim land as possible 

4) "be comforta"ble at most depths in the sea 



60. The complex machinery and great care used hy men when they go 
down into the sea shows that 

1) the Law of the Limits applies to them too 

2 ) it is difficult to imitate fish 

3) the sea is always in motion 

h) fish can disobey the Law of the Limits more easily than men 



61. On most smoggy days, there is a layer of very warm air over 

cooler air near the ground. From what you have learned in the 
movie, you would call this wall between warm sind cool air a 

1) Law of the Limits 

2 ) system 

3) thermo dine 

4) wave 



The four basic needs of animals and plants in the sea are: 



• 

63. 


li^^t 


J 


64. 


warmth 




65. 


safety 


• 



(in any order) 
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66. Each plant or animal has its own needs; "because of this, 

1) each form is usually found in a particular place 

2 ) plants and animals have the same needs 

3) plants grow everywhere in the sea 

k) fish and plants both need light to live 

67. A scientist wishes to use sunlight to li^t his aquarium, but 
he wishes to keep the dangerous warm rays of the sun away from 
the fish who live there. To do this he might 

l) shine the sunli^t through water to let only the cool rays 
reach the aquarium 

2) put ice in the aq.uarium to keep the fish cool 
3^ keep the water in the aquarium from moving 
k) change the Law of the Limits 
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APPENDIX D 
QUALITATIVE MEASURES 



Name 

Last Middle First 

School 

BE,, SURE TO,,M/^ YOUR.lttRK, 

,,,,, ,HEAVY„ANP,, BLACK,,,:, 

„„, ERASE,, COMPLETELY ANY ,ANS«pS ,„„ 
YOU WISH ,TD CHANGE, ,„„ 




THE STORY YOU SAW ABOUT 
THE ECO- SYSTEM. 



U. .ZIIZ z:zzGOOD: 

"^iWORK= 

N ::::: :.-=z =^GRUEL= 

A ..-zFAS!T 



3::z .l=r 




3:zz =r:=z =i=r= ^i-- 



Q:.. ..... zl..-. .... ^... 



.Q.-. .z-.z zlzzz zzzzz z^zz 



^.z zlzzz ...z z^z. 



^.z ..... ziz. zzzzz z2.z 



z^zz ===== bad ^ 



=}== ===== PLAY 



=3== ===== rind = - = 



-3== SLOt - 



E ===== ===== ===== STRON& 

A ===== ===== ====PLEASAN?t 

H ===== repetitive 



3=== ===== ^.z. ===== =!=== 



3 === ===== 2 === ===== zlzzz 



3=== ===== iz.-. ===== =1=== 



#== ===== =1== ===== =2=^ 



=0== ===== - i -- ■ 2-- 



=0== ===== =4== ===== =2^= 



=3== ==== WEAK===== ==== 



z^- ===== UNPLEASANT 



zz^z .==== VARIED= ==== 



====FEMININE 



5 ===== ===== ===== ====LOO& 

S 




===== ==::: ===== ACTIVE 
===== ===== ===== ==|ffiAV¥- 



3 === =:=:= 2 === ===== = 1 ==. 




3=== t" 

3=:: ===== ^.zz ===== =t== 



= 0 == ===== ==!== ===== ^ 

0 = == T = 

0 = - 1 - - == ^ 

= 0 == ===== ==!== ===== = 2 == 



= 3 == ===== mASCULINE== 

=. 3 = ===== SOFT===== === 

== 3 = ===== PASSIVE == 
== 3 = ===== LIGHT:- : : 



CIRCLE TEffi BEST ANSWER (h Mins) 



1. Did you feel that the story about the e co-system of the sea was 

? 



a) very long b) long c) OK d) short 



e) very short 



2. How did you feel about the story when it was shown to you? 

a) liked it b) liked it c) Just OK d) disliked e) disliked it 
flot a little It a little very much 



3 Which part did you like best ? 



4. Which part did you like least ? 



5 Did you have any trouble understanding it? Mark an X for Yes 

off o^No . If you have trouble, tell what this trouble was 

in the space below. 



6. How much do you think you learned? 
a) very little b) a little c) Just 

7. Was the test ^ 

a) very easy b) easy c) OK • 



OK d) a lot e) a great deal 



d) hard e) very hard 



8. If you used a workbook during the story, how did you feel that it 
helped you to learn? 

a) not at all b) a little c) Just OK d) a lot e) a great deal 
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